construction and validation of a low-cost laparoscopic

Construction and Validation of a Low-Cost Laparoscopic SimulatorF. Mandolfino(a) M. Sguanci(b), M. Minuto(c) ,G. Vercelli(d) ,G. Marcocci(e), M. Frascio(f)

(a),(b),(c),(e),(f),DISC - Department ofgeneral surgery(d)DIBRIS - Department of informatics, bioengineering, robotics and systems engineering

(a)[email protected], (b)[email protected], (c)[email protected], (d)[email protected],

(e)[email protected], (f)[email protected]

ABSTRACT

INTRODUCTION. To present the University of Genoa Advanced Simulation Center (ASC) and the design a trainer (eLap4D) that would achieve the equivalent goals of the fundamentals of laparoscopic surgery trainer at an economical cost. The validation process is going to be shown too. METHODS. The laparoscopic trainer is a physical low-cost laparoscopic training platform that reproduces the tactile feedback (eLaparo4d) integrated with a software for virtual anatomical realistic scenarios (Unity3D V 4.1). A sample of 20 students was selected, divided into 2 homogeneous groups with respect to the level of confidence with the use of video games, consolles, smartphones (this has been possible thanks to the use of a questionnaire, administered before the practical phase of training). The groups participated in a training program based on 5 basic laparoscopic skills (laparoscopic focusing and navigation, hand – eye – coordination and grasp coordination). So, a second and third study sample was chosen, consisting of 20 post graduate students (intermediate group) and 20 experienced surgeons; for theese groups was provided a training program identical to the previous group as well as their subdivision into 2 group. The face validity was used for an ergonomic analysis of the simulator, the construct to test the system's ability to differentiate potential expert users (experienced surgeons) from non-experts (student without experience in laparoscopic surgery).

RESULTS. We analyzed the results of the three samples obtained by comparing variables such as:

score % of fullfillment panality time

At the same time, the students’ improvements have been monitored, developing a customized learning curve for each user. To evaluate the structural characteristics of the simulator a specific questionnaire has been used. The results encouraged us. The simulator is ergonomically satisfactory and its structural features are adapted to the training. The system was able to differentiate the level of experience and also has therefore met the requirements of "construct validity". CONCLUSION. Valid laparoscopic simulators can be constructed at an economical cost.

Keywords: low cost simulation, face validity, construct validity, training, laerning curve

1. Background

The mission of a simulation center in medical reality is to improve patient safety and clinical outcomes by integrating medical simulation based teaching methodologies into the educational curriculum for all students, residents, attending physicians, nurses and other ancillary health care staff. The main goal of the Simulation Center is to improve safety within patient care. Current and future health care professionals “practice on plastic” honing their skills, refining advanced techniques and learning valuable social interactive tools for delivering important news to patients. This translational research becomes vital for creating the gold-standard in patient safety and medical teaching. One of the most interesting expericnce is about a completely original laparoscopic trainer. Nowadays laparoscopic surgery is considered the gold standard to treat a lot of diseases, but not all surgeons have acquired the skills necessary for laparoscopic procedures, for example such as proficiency in ambidextrous maneuvers with new tools, enhanced

Proceedings of the International Workshop on Innovative Simulation for Health Care, 2016 978-88-97999-81-2; Bruzzone, Frascio and Novak Eds.

1

hand-eye coordination, depth perception and compensation for the camera angle, the need to repeat the same exercises to improve these laparoscopic skills has made basic laparoscopy amenable to simulator based training. The continually increasing demand of more complex laparoscopic simulators has led to a rise in prices of these tools and has inspired us the creation of a 4d simulator which is a physical low-cost laparoscopic training platform that reproduces the tactile feedback (eLaparo4d) integrated with a software for virtual anatomical realistic scenarios (Unity3D V 4.1). The School of Medicine of Genoa and the Biomedical Engineering and robotic department (DIBRIS) have cooperated to create a low-cost model based on existing and brand new software. The simulator allows the team work: two surgeons can work together like in reality and the system allows the use of real operative instruments, all equipped with tactile feedback. But before using this simulator to assess skills and competencies it needs to be seriously and thoroughly validated: among the five validities Recognized (content, face, construct, concurrent and predictive) we has decided to employ the face validity and the construct validity: the first is usually used informally to define the realism of the simulator or whether the simulator represents what it is supposed to represent and the second because mandatory in distinguishing the experts from inexperienced operators based on a performance score. In this paper we are going to describe the platform validation results using these two types of validities: face and construct validity. 2. Materials and methods 2.1 The Advanced Simulation Centre The ASC has been introduced in October 2011 from the need to offer students and graduate students of the School of Medical Sciences and Pharmaceutical more adequate professional training to the health needs. The strongest motivations for the use of simulation for the training of future health professionals are: need to train to perform safety maneuvers increasingly complex and invasive; need to reduce the learning time curves of innovative procedures; increase in the medical-legal litigation and the carrying out of clinical maneuvers on the part of students must take place after an appropriate training that allows to learn from mistakes; introduction on the market of more sophisticated devices for the simulation that allow to reproduce more and more realistic clinical scenarios. The ASC has been designed on the model of the Simulation Centre in Montreal at McGill University and is organized into sections:

• Macrosimulation: using mannequins to the whole person or body parts that, depending on the technological complexity of the dummies and the complexity of clinical scenarios to be played, it is divided into high- medium-low fidelity.

• Microsimulation: consists of computer workstations for the solution of interactive clinical cases in order to train students and make clinical decisions correct in the right time.

• Relational simulation: using clinical environments realistically reproduced in which, using the technique of the game of roles and the use of "standard patients" students are trained to report the relationship with patients and working in a team.

• Virtual Reality: consists of devices with different technological complexity, from box trainers in computerized tools that can restore the sense of touch, by which you acquire manual skills such as basic surgical techniques or performing complex laparoscopic procedures.

The ASC is divided into two wings involving an area of about 400 sq. The curricular courses offered are as follows: Bachelor of Science in School of Medicine

• First Aid (first year): 270 students divided into 14 groups for 8 hours at group

• Biophysical and Clinical methodology (III year): 280 students divided into 32 groups for 20 hours in group

• Gynecology and Obstetrics (V year): 240 students divided into 8 groups for 8 hours at group

• Radiology (IV year): 240 students divided into 12 groups for 20 hours in group

• Pediatrics (V year): 240 students divided into 8 groups for 8 hours at group

• Emergencies (VI year): 240 students divided into 32 groups for 44 hours in group

• Vocational training medical-surgical (VI year): 240 students divided into 32 groups for 20 hours in the group.

Bachelor of Science in Nursing

• Vocational training (II, III): relational workshops for 80 students of the pole S. Martino divided into 4 groups for a total of eight hours per group.

• Check certification service according to the method OSCE (Objective Structured Clinical Evaluation) for 470 students of all regional poles (8 hours for 20 days)


2

• Bachelor in Physiotherapy • Aesthetics of passive mobilization (II and III

year): 40 students divided into 4 groups for 4 hours in group

Bachelor of Science in Dental Hygiene

• Relational Laboratory (III year): 40 students divided into 4 groups for 4 hours in group

Bachelor of Science in Dietetics

• Relational Laboratory (III year): 20 students divided into 2 groups for 4 hours in group

Specialization schools

• Anesthesiology and Intensive Care • Emergency Medicine • Internal Medicine • Cardiology • Nephrology • Gynecology • General and Digestive Surgery

We can estimate a commitment to teaching between 50 and 20 hours/year for each school. The total number of students trained annually at the center has been estimated at around 2800 for a total of about 2500 hours of teaching imparted. Students, on request, may attend the center individually or in small groups to self-study in the free zones of the programmed teaching. At the end of each course, every student receives a quality assessment questionnaire perceived. The ASC also hosts and organizes courses aimed at external users, health professionals for their continuing education or categories of citizens who for various reasons are related to health issues. It is, therefore, offered to BLSD Courses (Basic Life Support and Defibrillation), ATLS (Advanced Trauma Life Support), First Aid, CRM (Crisis Resource Management), etc. According to the ARS (Regional Health Agency), the Coordination of Rare Diseases and ATM Rare Diseases of IRCCS Gaslini pediatric hospital, at the Centre are held free courses for caregivers, family, or household employees of patients, especially children, suffering from chronic diseases disabling, in order to reduce the Hospitals and ensure a more safe and comfortable home care. These courses have a semi-annual basis and to date have been over 150 Caregivers formats. For each type of training activity in the simulation it uses the teaching methodology derived from the training of flight personnel, now adopted by decades by the Aeronautical Companies. In particular, each procedure is broken down into a check-list of actions that learners construct with the guidance of the tutors in

order to acquire the necessary automatism in critical or emergency conditions. The evaluation of 'learning takes place using the method OSCE (Objective Structured Clinical Examination), consisting of a "stations" exam where the student must perform the procedures using the check-list he built. It’s already on a successful attendance of the center by the students of Biomedical Engineering courses and Bioengineering of the Polytechnic School. The center offers these students the opportunity to make contact with simulators of the latest generation and to study the materials they are made of is the software that govern them, the video-recording system and audio connection between systems of the center and between the center environments and operating rooms IRCCS San Martino, in the simulation devices for measuring air quality and dispersion of medical gases and fumes of which are equipped the rooms of Macrosimulation to high-fidelity. Currently, in collaboration with the center, they are being developed some degree thesis in Bioengineering. The ongoing collaboration with the Polytechnic School has also resulted in the implementation by the DIBRIS a prototype augmented reality for video-laparoscopy with haptic properties, being validated by doctors in specialist training of the surgical field schools. 2.2 The simulator system The development of a laparoscopic surgery simulator is nowadays one of the most important subjects in the field of MIS. There are many important aspects that need to be considered when designing such a kind of simulator. Surgical simulators are very complex systems, mainly because MIS techniques are characterized by a very high level of visuo-motor coordination and multi- user cooperation within reduced operating spaces. Moreover, the implied technologies are sophisticated and expensive. Laparoscopy simulators are a powerful way to improve the skill gamut for the medical doctors to master such systems at a very low cost. Developing a simulator with this kind of purpose is not easy and has many concerns to take into account. For instance, other than having a well structured visualization environment, aspects such as information communication, feedback capabilities, human factors, operative constraints, ergonomics and training aspects need great attention. Although at the current state not all of them has been fully developed yet, the whole design process of the project, including the choice of hardware and software technologies, has been specifically approached to reach the introduced goals. The system is based on a nodejs application server that manages the visualization system, the communication with hardware interfaces and the database where users’ data are stored. The server technology is indeed a sort of data gateway between the several different elements,


3

regardless they are hardware or software. Figure 1 shows how communication data are exchanged from the very low part of the system (Hardware Interfaces, bottom) to the user interface (HTML Client,top).

Figure 1. part of the system simulation The user interface is a simple HTML5 web page running a Unity3D engine 2 plugin. We run several performance tests to compare Unity3D and native WebGL, getting same results. We finally decided to adopt Unity3D engine due to its rapid development time. WebGL is a great technology but still too young to allow us working on a powerful and robust framework. The use of web pages as the main user interface allows us to be more versatile and in the future will give us the possibility, thanks to HTML5 powerful characteristics, to easily share contents in a live way with other systems. An interesting feature is, for example, having the possibility to be guided by an external supervisor, who is monitoring the training phase, while data are quickly exchanged via internet. A. Visual and Physical Modeling As previously introduced, visual modeling is a very important aspect of the entire project. A laparoscopic surgery simulator needs a detailed representation of the organs and the tissues inside of the human abdomen. The meshes included in eLaparo4D are developed in Blender 3D Modeling software3, and then imported in Unity3D, including textures and UV maps. Eventually, in Unity3D render shader materials are added to the raw meshes, to simulate the specific surface of each of the modeled tissues. In Figure 2, a screenshot of the current virtual environment is shown.

Figure 2: a screenshot from the current aspect of the virtual environment compared to a screenshot of the camera view of a real surgical operation A great effort has been made to realistically simulate

physics avoiding system overloads (excessive computational loads, affecting usability). As remarked by our colleagues of the Laparoscopy Unit of the Department of Clinical Surgery, highly specific training sessions are required to help the operator achieving a proper skill set. In an ideal scenario, medical students should have access to a complete simulator composed of several training scenes, as part of a modular and step-based training process. While the main components and controls of the simulator should be in common, each scene should focus on a very specific surgery operation, differentiating in: the zone and the organs physically manipulated (the target), the particular surgical maneuvers performed (the task), and the type of manipuli used (the means). This implies that, according to these 3 components, not all the elements included in a training scene need the same level of realism, especially in terms of physical behavior. In general, the targets of the operation are supposed to have a more accurate physical behavior with respect to an organ in background; but also among the targets the level of accuracy can vary. Even in the same scene the physical simulation of the targets can change over time, according to the manoeuvre the operator is currently performing with the manipuli (e.g. a simple grasping vs. a precise carving). Furthermore, the learning of a complex task - carried out with complex means - should be achieved subdividing the task itself into several simpler steps, preparatory to a complete simulation; so, often, the global complexity of the physical simulation of the same set of organs can vary from scene to scene. Considering these remarks, we developed a dynamic parametric physical simulation approach, arbitrary applicable to the rendered meshes in every scene and able to avoid system overloads. Such an approach permits the creation of different scenes starting from the same set of models and interaction algorithms, easily supporting a step-based training. In detail, each 3D object in the scene carries a selectable 3 layer collider component, driving a vertex deformation script. The first layer is a simple box collider; the second one is a combination of simple shape colliders which cover, with good approximation, nearly all the volume of the object; the third is a precise mesh collider which exactly coincides with the vertex disposition of the object’s mesh. In Figure 3 it is possible to see the 3 different collider layer for a gallbladder model. According to the relevance the 3D object has in the scene (depending on target, task and means of the currently simulated operation), one of the 3 layer is activated, modifying the physics behavior defined in the vertex deformation script. When the box collider layer is active, the script handles collisions, allowing motion but not modifying the aspect of the colliding objects. This configuration is proper for background objects, far from the target. The second, composite, layer supports a more precise collision detection and introduces a script-based rough surface deformation when simple collisions occur (e.g. two organs collide while one is grasped and moved by the operator). This configuration provides a level of


4

realism suited for the organs that surround the actual target of the operation, or for the targets undergoing simple manipulations (simple tasks like flipping, pushing, lifting, etc.). Finally, the mesh collider layer allows the deformation script to perform a precise local vertex deformation, whenever a collision is detected. Such a detailed behavior supports inward surface deformation caused by pressure, as well as tissue stretching, folding and cutting, typical of manipuli-based surgical manoeuvres. Indeed, this configuration fulfills the strong needs for realism of the targets of the surgical operation, especially when the task is demanding and complex manipuli are the main means. The use of each layer is characterized by a different computational load: light for the first layer, intermediate for the second one and heavy for the third one. The load, obviously, depends also on the level of detail of the modelled meshes. In addition, the chosen layer can be switched dynamically. This means that the same organ can have a more or less accurate physical response to manipulation, according to the evolution of the system (e.g. the operator’s activity, the currently interacting manipulus), limiting as much as possible the CPU load while preserving realism. Moreover the real-time setting of the layer, coupled with the monitoring of the operator’s performance, offers the intriguing possibility to dynamically adjust the complexity of the task, automatically choosing a level of realism that fits or challenges the operator’s skill. Using a Unity stereoscopic plugin, we are able to visualize the scenes in stereoscopic 3D. The possibility to train operating with a stereoscopic visual feedback engaged our colleagues of the Laparoscopy Unit of Clinical Surgery Dept., since stereo cameras for real laparoscopy have been accurately assessed in biomedical engineering research, and are quickly spreading in the medical industry.

Figure 3. different collider layer for a gallbladder model 2.2.1 Haptic Feedback Haptic feedback is implemented thanks to the use of three Phantom Omni devices from Sensable. The first two are used as manipuli (grasper, hook or scissors) and the third one is used to move the camera within the virtual abdomen, as it happens in a real scenario. The system generates a resultant force when the user puts a manipulus in contact with a mesh, according to the executed task. Phantom devices have been chosen because reasonably low cost although precise enough

for the needed level of realism. Furthermore, their stylus-like shape will permit a complete merging of the devices with the physical environment reconstruction; in particular, each stylus will be easily connected to real manipuli. Thanks to an Arduino board connected to a vibrating motor we have also included a vibration feedback. Vibration is used to enhance the realism of operations like tissue shearing (hook) and cutting (scissors). The current feedback solution, coupled with the web- based structure of the application, makes available a haptic- based remote guidance, during which a supervisor, even if not physically present in the same room, can haptically guide the hands and the manipuli of the trainee to show him/her the proper way to execute a critical task. At the moment we are using a very basic force feedback calculation algorithm, based on compliant contact. Although a much more detailed force feedback solution is needed, at the current state eLaparo4D allows a first comparison with real laparoscopy systems and permits the collection of feedback from medical doctors. 2.2.2 The training interface Training is a key aspect in the eLaparo4D system as already outlined. The user has his/her own profile, is tracked over time and has is/her own history. Every exercise has its own allowed/not allowed actions letting the user earn or loose points. The scale and points assigned have been decided with the consultancy of several medical doctors, giving us feedback about what are the needs of a well skilled surgeon. The possibilities given by a HTML5 user interface allow us to be very versatile in the user profile management. 2.3 The validation process A valid simulator measures what it is intended to measure. There are a variety of aspects to validate; subjective approaches are the simplest. In this sense, we have chosen 2 different kind of validation: 1. The Face Validity 2. The Construct Validity Face validity usually is assessed informally by no experts and relates to the realism of the simulator; that is, does the simulator represent what it is supposed to represent. This kind of validity relates to the realism of the simulator. A questionnaire validation was created. In this document 12 closed-ended questions were selected about the following topics:

• ergonomics • structure


5

• realism • tactile feedback • quality

For each question must be given a score according to the rating scale "Likert" (Highly inadequate, Insufficient, Sufficient, Good, very good). Concurrent validity: is the extent to which the simulator, as an assessment tool, correlates with the “gold standard.” This testing can be achieved by evaluating 3 groups of subjects, with a different professional experience, with the simulator, comparing different variables. This necessitates establishing an objective structured assessment of technical skills (OSATS) evaluation by which the model or “gold standard” performance can be assessed reliably for comparison.(Max V. Wohlauer et al. , 2013) About this, the simulator must be able to distinguish the experienced from inexperienced surgeons. This is best determined by testing a large number of surgeons with various degrees of training, experience, and frequency of performance of a specific surgical skill or procedure. For competency assessment, the performance of an individual on a simulator should ideally predict, or at least correlate with, that individual’s performance in the real environment of the operating room. As such, a valid and reliable measure of operating-room performance must be established. This allows differentiation between surgeons assumed to be clinically competent (experienced or expert clinicians) and non competent (junior or inexperienced residents). 2.3.1 Sample and inclusion criteria We have involved a total of 60 subjects to the validation program. This entire group is divided into 3 categories: cluster A is composed by 20 students of Medical and Pharmaceutic Sciences of the University of Genoa without any experience in laparoscopic surgery, cluster B by 20 general surgery residents with moderate experience in surgical skills and Cluster C by 20 experienced surgeons but not peculiarly in laparoscopy. About “Selection criteria” we have chosen the number of laparoscopic surgical procedure as first operator as parameter.

• Group A: novices (NO experience in laparoscopic surgery)

• Group B: 20 intermediate (at least 10 total laparoscopic operations in the last year)

• Group C: 20 experienced (more than 10 total laparoscopic operations in the last year)

Each group has been divided into two smaller homogeneous groups based on the questionnaire about the personal level of confidence in the use of videogames, virtual platforms, etc:

• Subgroup A1, B1: little/absolutely not confident

• Subgroup A2, B2: confident/very confident

The questionnaire has been administered to each subject before the beginning the test. 2.3.2 Testing mode and setting To guarantee a correct statistic analysis, we have adopted a closed testing system where the subjects had a limited number of attempts (an open testing system might show bias like weakness, time delays or methodological limits). When finished the test, the beginner and intermediate groups have been completed the “Face validity” questionnaire to explore the ergonomic adequacy of the system. Each subject had max two attempts for every examination (2 attempts for exercise 1 level easy, 2 attempts for exercise 1 level intermediate, 2 attempts for exercise 1 level difficult). Each participant has finalized 6 examinations for a total of 30 at the end of the process. The setting has been the same during all the parts of the process. To increase the subject 's perception of the scenario in which it will operate, every subject had to dress surgical gloves, coat, mask and headdress. Similarly, the platform has been prepared with the virtual utilities present on the surgical field to make the hand pieces movements more adherent to reality. 2.4 Basic skills For the platform validation, 5 tasks have been selected. These exercises are related to the acquisition of tasks which allow students to reach basic gestures competences. They could practice using probes that simulate the haptic feedback according to the kind of action.

The 5 selected tasks are:

1. laparoscopic - focusing - navigation: This task

aims to evaluate the ability to navigate a laparoscopic camera with a 30º optic. This is done by measuring the ability to identify 14 different targets placed at different sites Two different exercises were chosen: Exercise 1: the student, working with a 30° ptic, have to focus different solid targets in a static scenario. This task evaluates the macro – focusing. Exercise 2: the student working with a 30° ptic, have to focus a lot of hidden micro- targets, placed in different areas of the scenario.


6

Figure 4: a screenshot of task 2

2. hand – eye – coordination (HEC): This task aims to evaluate the ability to work with the non-dominant and dominat hand. The camera is static. Two different exercise were chosen: Exercise 3: the student have to touch a defined point in an “circular target” with the left and right instrument simultaneously


Exercise 4: the student have to touch a lot of spheres that appear sequentially and in random positions. There is a time limit to center and touch each sphere with the right and left hand. In this task, the camera is static.

Figure 6: a screenshot of task 4 Exercise 5: the student have to grasp 3 objects and to put these in a selected form.


For each of these tasks, a certain number of metrics have been automatically recorded. Metrics are defined as follows: – Total time. Time that the user needs to

accomplish the task – Fulfillment. Percentage of partial tasks done

within the established time. – Penality: number of penality about each task. – Score: task’s score – Coordination – Accurancy

Which metrics are recorded for each task is shown in Table 1.

Task Description Metrics Navigation ability to

navigate a laparoscopic camera with a 30º optic

Fulfillment (%) Total time (s) Score penality

Navigation and focusing

the student have to focus different solid targets in a static scenario


Coordination (HEC) 1st exercise

the student have to touch a defined point in an “circular target”

Fulfillment (%) Total time (s) Score Penality Coordination Accurancy

Coordination (HEC) 2nd exercise and 3rd exercise

the student have to touch a lot of spheres that appear sequentially and in random positions. The student have to grasp 3 objects


Table 1 “Metrics and Tasks” in the Construct Validity


7

2.7 Setting

The setting has been the same during all the parts of the process. To increase the subject 's perception of the scenario in which it will operate, every subject had to dress surgical gloves, coat, mask and headdress. Similarly, the platform has been prepared with the virtual utilities present on the surgical field to make the handpieces movements more adherent to reality.

2.8 Data analysis

We have collected for each group several variables about the level of confidence with virtual platforms, and data about execution time, score, penalty where applicable, motion accuracy where applicable, motion coordination where applicable.

2.9 Face validity questionnaire

All Expert and intermediate subjects were requested to fill a Face validity Questionnaire, referred to characteristics of the eLaparo4D simulator (11 questions). The questions had to be answered in a 5-point Likert Scale:

• Strongly disagree • Disagree • Neither agree nor disagree • Agree • Strongly agree

2.10 Statistical Analysis The results are expressed as mean ± standard deviation,

median, minimum/maximum values, and percentages.

The Shapiro-Wilk test was applied to evaluate the

normal distribution of continuous variables. Differences

for answer scores between validation and reference

group were evaluated using the Wilcoxon-Mann-

Whitney test. The Spearman’s rho was used for

correlation analysis. A two-tailed P value ≤0.05 was

assumed for statistical significance. Statistical analysis

was performed using the R software/environment

(version 3.2.5; R Foundation for Statistical Computing,

Vienna, Austria).

3. RESULTS AND DISCUSSION

3.1 Results 3.1.1 Face Validity

The questionnaire analysis (submitted to experts and intermediate) allowed to obtain the following data: Experts opinion: • A real confidence in the ability of this device to

allow an accurate performance measurement (4 + 0,81)

• A great degree of realism in the management of the optic in the virtual scenario (3,9 + 0,87)

• An excellent realism of targets (4,1 + 0,56) • An excellent degree of realism of the positioning

of the instruments (3,9 + 0,56) • An high quality of the images (4 + 0,81) • A great Haptic feedback (sensation) (3,3 + 0,67)

Excellent degree of usefulness of simulation in reference to 'acquisition of skills, "basic" hand-eye coordination (4,4 + 0,69)

Intermediate opinion: • An excellent degree of realism in the management

of the 30° optic • A great quality of scenario • A very good capability of the simulator to teach

gestures and action • The devices position show a good degree of

realism

Characteristics Experts (n=12) Realism 3,6 + 0,84 Degree of realism of the positioning of the instruments

3,9 + 0,56

quality of the images 4 + 0,81 Realism of targets 4,1 + 0,56 Degree of "realism" movement 3,4 + 0,96 Haptic feedback (sensation) 3,3 + 0,67 Degree of realism in the management of the optic 3,9 + 0,87 Degree of utility of the haptic feedback 3,5 + 0,70 Degree of usefulness of the simulator about acquisition of "basic" skill (hand-eye coordination)

4,4 + 0,69

Degree of usefulness of the simulation about acquisition of skills with non-dominant hand

3,9 + 0,63

Degree of overall usefulness of the simulator about acquisition of basic laparoscopic techniques

3,8 + 1,03

Confidence in the ability of this device to allow an accurate performance measurement

4 + 0,81

Table 2 Face Validity (expert) Questionnaire results 3.1.2 Construct validity

About construct validity, there were significant differences between the expert group, the intermediate group and the novices group in several tasks, while a difference non statistically significant has been appeared between experts and intermediates.


8

The tasks 3, 4 and 5 (about coordination) discriminates between experts and novices in all the evaluated parameters. There were significant differences between the experienced group and non-experienced group in the task 3, in terms of “total time”, “score”, “coordination” and “accurancy”; this task shows a better executions accomplished by experts than the ones accomplished by novices. The task 2, about navigation, shows a better percentage of fulfilment in favour of expert group (90/100% fulfillment). Total time, shows significant differences in task 2,3,4,5. There weren’t significant differences between the experienced group and non-experienced group in the task 1. As previously described in the methodology, metrics that are evaluated in all tasks are total time, fulfillment, score and penality.

3.2 Discussion

The advent of new surgical methods and devices, such as endoscopic, laparoscopic and robotic surgery, caused the need for systematic skills training in an efficient and safe environment. There are several commercial LTBs and VRSs that analyze the volume, distribution, economy, angle and smoothness of instrument movements, and give numerical and/or statistical results to the trainee after completion of the task(s). However, these devices are generally expensive, and not every center can afford to incorporate them into their education curriculum. Besides these expensive training boxes/ simulators, some authors have developed themselves either low-cost laparoscopic simulators with reasonable budgets. Starting from this point, our group has developed a low cost laparoscopic simulator for basic skills and for cholecystectomy. We have reported the results of the validation process. The validations’ steps of these kind of training system are essential in order to determine their capacity for surgeons training although as far as we know, there is not any mandatory validation strategy. The Face validity and the Construct validity are two of the more reported in literature. The Construct validity determines the capacity of the simulator to punctuate the execution according to the level of experience of the subject who is accomplishing the task. So, a construct validated simulator will be able to distinguish between surgeons with different levels of experience in laparoscopic surgery. The Face Validity is just based on the opinion and experience of surgeons and cannot be used in every case to define the validity of a new simulator. As the face validity is very subjective, it is usually used at the first stages of validation. The aim of this work is to validate “eLaparo4D” simulator accomplishing a face and construct validity in

order to determine whether it is adequate for basic skills training. Expert group and intermediate group agree with usefulness of the simulator in reference to 'acquisition of skills, "basic" hand-eye coordination and confidence in the ability of this device to allow an accurate performance measurement. The realism of the targets and the scenario is a great characteristic, like the position of the instruments. The haptic feedback is considered by expert as acceptable, most important elements in this kind of virtual simulators. The results of the study show that there are significant differences between the tasks execution by novices and by experts and intermediates for the evaluated metrics. Among all, navigation and coordination tasks show the clearer results. The task 1 about navigation not present any difference between the different levels of experience: this result can be due to the fact that novices have experience virtual games and in video camera use. In task 3,4 and 5 the difference between novices and experts is evident; total time, score and penalty are in favour of experts. In task 3, the expert group showed a better coordination and accurancy than novices. The “total time” are evaluated in all tasks because is an important variable; novices need more time than experts to finish the tasks in all cases and experts fulfil the majority of the tasks and more efficiently than novices. To evaluate the reliability, we decided to perform the correlation index to the metrics: total time and score. The results of this test show an high value of correlation for the total time and a lower value for the score. From these values, the Split half Methodology was applied, to calculate the coefficient of Reliability; we applied the Spearman-Brown correction and the final result was: 0.91 The thin difference between intermediates and experts in several tasks could probably be explained in the definition of “experts”. These subjects revealed a strong surgical experience but not peculiarly in laparoscopy. This conclusion leads us to the point that eLaparod4D could be used in training programs as an assessment tool.

ACKNOWLEDGMENTS

We would like to thank G. Vercelli, L. Lagomarsino, E. Bellanti and S. Marcutti of Dept. of Informatics, Bioengineering, Robotics and Systems Engineering for their precious collaboration. Thanks to all of our collaborators.

REFERENCES

1. htpp://www.nodjes.org 2. htpp://unity3d.com


9

3. htpp://sensable.com 4. htpp://blender.org 5. Max V. Wohlauer, Brian George, Peter F.

Lawrence, Carla M. Pugh, Erik G. Van Eaton, Debra DaRosa. 2013. Review of Influential Articles in Surgical Education: 2002–2012. Journal of Graduate Medical Education 5:2, 219-226.

6. Gallagher AG, Ritter Em, Satava RM (2003) Fundamental principles of validation, and realiability: rigorous science for the assessment of surgical education and training. Surg Endosc 17:1525– 1529

7. Molina CR, de Win G, Ritter O, Keckstein J, Miserez M, Campo R (2008) Feasibility and construct validity of a novel laparoscopic skills testing and training model. Gynecol Surg 5(4): 281–290


10

construction and validation of a low-cost laparoscopic

Documents