development of a behavioural marker system for scrub practitioners' non-technical skills...

Development of a behavioural marker system for scrubpractitioners’ non-technical skills (SPLINTS system)jep_1825 1..7

Lucy Mitchell MA MRes,1 Rhona Flin BSc MSc PhD CPsychol FBPsS FRSE,2

Steven Yule BSc MSc PhD,3* Janet Mitchell RGN DipN MScN,5 Kathy Coutts RSCN6 andGeorge Youngson CBE MBChB PhD FRCPE FRCSEd4

1Research Fellow, 2Professor, 3Lecturer, 4Professor Emeritus, School of Psychology, University of Aberdeen, UK5Practice Education Facilitator, Aberdeen Royal Infirmary, UK6Theatre Nurse Manager, Royal Aberdeen Children’s Hospital, UK

Keywords

marker system, non-technical skill, nurse,scrub practitioner, SPLINTS

Correspondence

Dr Lucy MitchellSchool of PsychologyUniversity of AberdeenAberdeen, AB24 3UBUKE-mail: [email protected]

* Steven Yule is now at STRATUS Centerfor Medical Simulation, Brigham &Women’s Hospital/ Harvard Medical School,Boston, MA.

Accepted for publication: 23 December2011

doi:10.1111/j.1365-2753.2012.01825.x

AbstractRationale, aims and objectives Adverse events still occur despite ongoing efforts toreduce harm to patients. Contributory factors to adverse events are often due to limitationsin clinicians’ non-technical skills (e.g. communication, situation awareness), rather thandeficiencies in technical competence. We developed a behavioural rating system to providea structured means for teaching and assessing scrub practitioners’ (i.e. nurse, technician,operating department practitioner) non-technical skills.Method Psychologists facilitated focus groups (n = 4) with experienced scrub practitio-ners (n = 16; 4 in each group) to develop a preliminary taxonomy. Focus groups reviewedlists of non-technical-skill-related behaviours that were extracted from an interview study.The focus groups labelled skill categories and elements and also provided examples ofgood and poor behaviours for those skills. An expert panel (n = 2 psychologists; n = 1expert nurse) then used an iterative process to individually and collaboratively review andrefine those data to produce a prototype skills taxonomy.Results A preliminary taxonomy containing eight non-technical skill categories with28 underlying elements was produced. The expert panel reduced this to three categories(situation awareness, communication and teamwork, task management), each with threeunderlying elements. The system was called the Scrub Practitioners’ List of Intra-operative Non-Technical Skills system. A scoring system and a user handbook were alsodeveloped.Conclusion A prototype behavioural rating system for scrub practitioners’ non-technicalskills was developed, to aid in teaching and providing formative assessment. This importantaspect of performance is not currently explicitly addressed in any educational route toqualify as a scrub practitioner.

IntroductionThe operating theatre is a high-risk environment where multidis-ciplinary teams are required to work together towards a commongoal: the safe surgery of the patient (see [1]). Research has sug-gested that adverse events occur in over 10% of patients admittedto hospital [2] despite ongoing efforts to reduce this since ‘ToErr is Human’ was published [3]. Adverse events are unintendedinjuries or complications caused by the management of thepatient’s care rather than the underlying medical condition [4] anda systematic review of adverse event studies suggested that

41% of all hospital adverse events occur in the operating theatre[5].

One important member of the multidisciplinary operatingtheatre team is the scrub practitioner (i.e. nurse, instrument tech-nician, operating department practitioner). He or she has manyresponsibilities including ensuring that the swab and instrumentcounts are carried out so that everything is accounted for at theconclusion of the surgical procedure [6]. However, specimens [7]and other foreign objects are still sometimes retained in patients[8] despite technical guidelines which should be followed [9,10].A study of medical records for malpractice claims in the United

bs_bs_banner

Journal of Evaluation in Clinical Practice ISSN 1365-2753

© 2012 Blackwell Publishing Ltd, Journal of Evaluation in Clinical Practice 1

States found that retention of a foreign object was more likelywhen there was an unexpected change in the procedure or inemergency rather than elective cases [8]. These findings suggestthat errors are more likely to occur when there is less time for thescrub practitioner to prepare or when there is increasing cognitivedemand in order to keep up with a dynamic, evolving surgicalsituation. Although the instrument count is a technical duty, it alsorequires non-technical skills, such as situation awareness and com-munication, in order to perform it reliably. Non-technical skills arethe cognitive and social skills that complement technical skills forsafe and effective practice [11]. These skills have only recentlybeen included in undergraduate medical curricula [12] and are notyet explicitly taught in any teaching curricula en route to qualifi-cation as a scrub practitioner.

Research by social scientists and clinicians has demonstratedthat the underlying causes of adverse events to patients are oftenattributable to failures in non-technical skills such as communi-cation [13,14] or teamwork [15,16] rather than a lack of technicaltraining or expertise. Methods taken from aviation for assessingnon-technical skills of pilots (NOTECHS; [17]) have led to thedevelopment of behavioural rating systems for anaesthetists’(ANTS; [18]) and surgeons’ (NOTSS; [19]) non-technical skills.Despite this research, and the increasing use of these systems forexamining non-technical skill performance of surgeons [20] andanaesthetists [21], the process of designing a rating system from aset of identified behaviours has only been partially described pre-viously (see [22,23]).

The development of the rating system for scrub practitioners’non-technical skills began with a review of nursing literature[24] and an interview study with experienced nurses (n = 25) andconsultant surgeons (n = 9) [25]. Those studies suggested that‘situation awareness’, ‘teamwork’, ‘communication’ and ‘copingwith stress’ were important skills for the scrub practitioner andgenerated lists of behaviours related to scrub practitioners’ non-technical skills. The next step in the process was to identify theessential categories, elements and behavioural markers for safeand effective scrub practitioner performance. This would enablethe development of a taxonomy suitable for use by scrub practi-tioners, to assist with training and assessing non-technical skillsin the operating theatre. The present paper describes that aspect ofthe development process of the Scrub Practitioners’ List of Intra-operative Non-Technical Skills (SPLINTS) behavioural ratingsystem.

MethodsTo guide development of the system, the authors – three psycholo-gists, two theatre nurses and a consultant surgeon, considered theusability requirements of this tool and agreed that the systemshould not be overly complex, resource or attention demanding,so that it could be easily incorporated into everyday practice inthe operating theatre. To achieve this, the following design criteriawere established, with reference to previously developed behav-ioural rating systems [22,26,27], so that the resulting prototypesystem was fit for purpose. The system should1 Focus on the skills that are observable from behaviour.2 Contain skills that are applicable to scrub practitioners in allsurgical specialties.

3 Be set as a hierarchical structure with three levels of description;category, element, behaviour.4 Use active verbs for skills and understandable language fordefinitions.5 Show a simple structure and layout with a rating scale that fitson one page so that it can be easily used in the operating theatreenvironment.6 Contain enough information in the handbook so that it is usableby scrub practitioners with limited additional training.

The present study was conducted in two consecutive stages;focus groups to discuss the lists of behaviours and label thenon-technical skills followed by a refinement process by an expertreview panel to reduce the number of categories and elements andaid usability. These stages will be described separately, and thendiscussed together in the Discussion.

Ethical approval was granted from the University of AberdeenSchool of Psychology and North of Scotland NHS Research EthicsCommittees (refs: pRGF/088/07; 07/S0802/57).

Focus groups

Design

The facilitated focus group approach is a recognized means bywhich competency frameworks can be developed [28]. The focusgroup approach encourages participants to talk to one another,exchange ideas and comment on personal perspectives [29]. Thedata from those discussions can be gathered by the researcherwho is facilitating the focus groups [30]. Stage one involved focusgroups (n = 4) of experienced scrub practitioners discussing andreviewing lists of non-technical skill behaviours extracted from aliterature review [24] and interview study [25]. Through an itera-tive process [31], the focus groups produced a comprehensive listof non-technical skills and elements that the efficient scrub prac-titioner should ultimately develop, as well as examples of goodand poor behaviours (i.e. markers) for those skills.

Participants

Participants were recruited via posters distributed around operat-ing theatres in the three participating acute Scottish hospitals.They were all female scrub nurses (n = 16), holding a theatrenursing team leader position, each with >10 years operatingtheatre experience (no further demographic data collected).

Materials

Eight lists of behavioural data items were produced for the focusgroup sessions. Table 1, column 1 lists the eight skill categories.These lists were compiled from the data extracted from an inter-view study [25].

Although the data were presented as lists of behaviours for eachof the eight non-technical skill categories, the underlying elementlabels [from the interview data coding process; (see [25])] hadbeen removed, for example, the list of ‘situation awareness’ datasegments comprised lists of behaviours, which had been previ-ously coded in the interview study as ‘listening’, ‘watching’,‘understanding’ or ‘anticipating’. During discussion, the focusgroups provided their own labels (which generally mirrored the

SPLINTS behavioural marker system L. Mitchell et al.

© 2012 Blackwell Publishing Ltd2

earlier labels) but this method was used to ensure that the elementlabels were in appropriate language for use in the perioperativeenvironment. For example, Table 2 shows an extract from the‘situation awareness’ skill list with four examples of previouslycoded data segments on ‘anticipating’. All the items were num-bered on the lists for ease of reference for the participants duringtheir discussions.

Procedure

The focus groups were facilitated by one or two of the authors(LM and RF). Sessions were conducted in a room within the

Table 1 Skill categories and the respective definitions given to focus groups for discussion with labels and possible behavioural marker examplessupplied by the respective focus groups

Skill (no. of items) DefinitionFocusgroup

Element label givenby focus group

Possible behaviouralmarker provided

Communication (95) Exchanging information, both verbaland non-verbal between scrub nurseand others in the team

1 Verbal (28) Keeps circulating nurse informedNon-verbal (17) Uses non-verbal communication when

appropriateAssertiveness (50) Gives clear and concise instructions

Teamwork (45) Co-ordinating activities with teammembers, supporting others andestablishing a shared understanding(about team members’ roles/ plan ofpatient etc.)

2 Sharing (5) Provides other team members withinformation

Clarifying (5) Asks if unsureCo-ordinating (23) Prioritizes tasks from other team

membersLeading (4) Supports others in teamTeam dynamic (8) Respects others in team

Leadership (12) Guiding others by providing directionand instruction while understandingand considering the roles and needsof others in the team

2 Guiding (12) If you are working with a junior or atrainee (surgeon) you can advise themwhat their consultant would normally do

Decision making (20) Identifying options, consideringoptions, selecting andcommunicating options,re-evaluating and reviewingdecisions.

2 Equipment (17) Hands correct instrument to surgeon

Staff (1) Allows circulating nurse to leave theatre

Timing (2) Interrupts surgeon, for example, withmessage appropriately

Task management(57)

Planning and preparing for the list/caseincluding organization of equipment,staff and other resources and;adhering to protocols and ensuringothers do likewise

3 Organizing (12) Checks preferences of surgeon

Planning and preparing (13) Plans for unexpected eventsEquipment (21) Ensures equipment functioning effectively

Maintaining standards (11) Follows protocols

Managing stress (34) Managing the effects of stressincluding pressure from the casecircumstances and created byothers in the team

3 Coping with pressure (8) Does not lose temperStress in self (12) Displays calm exteriorStress in others (14) Understands the skill mix in the team

Fatigue (10) Coping with fatigue including howtiredness affects performance

3 Fatigue effects (10) Recognizes when tiredness affectsperformance

Situation awareness(103)

Gathering information, understandinginformation, recognizing andunderstanding situations orbehaviours of others in the theatreteam, anticipating and projectinginto the future

4 Listening (16) Listens to conversations between otherteam members

Watching (19) Watches operating site/TV screen inlaparoscopic

Understanding (10) Recognizes change in patient condition/procedure

Anticipating (19) Remains ahead of surgeonAwareness (15) Ignores distractionsAtmosphere (4) Controls musicKnowledge (16) Demonstrates knowledge of individual

surgeonsTiming (4) Asks for supplies at appropriate time

Table 2 Extract of data segments from the ‘situation awareness’ listcoded in the interview study [25] as ‘anticipating’

1. We have the standard crates but you have to anticipate the otherinstruments they might need

2. I don’t like the surgeon to have to wait for things. I like to have itthere before they need it, so things run smoothly

3. . . . even if it’s coming slowly in a standard operation you are kindof thinking four steps ahead of everything

4. I suppose you’re pre-empting what the surgeon needs as you areanticipating what instruments you’ll need

L. Mitchell et al. SPLINTS behavioural marker system

© 2012 Blackwell Publishing Ltd 3

participants’ theatre suite. Each focus group session began withthe facilitator reading the study protocol to the participants. Thisinformed the participants that they were each to read the list/sof data segments individually before commencing the group dis-cussion. They were asked to discuss the skills that were beingdescribed in the list/s and to provide labels for those elements (e.g.‘verbal’ was labelled as an element of the ‘communication’ skillcategory). The sessions lasted 3 hours with two short refreshmentbreaks, and the participants were invited to do the following:1 Provide a preliminary list of observable non-technical skills forthe scrub nurse.2 Label the categories and group underlying skills into elementsof those non-technical skills.3 Develop some examples of observable behaviours for good andpoor performance of those labelled skill elements.

Focus group 1 discussed the list of ‘communication’ (n = 95)data segments while the ‘teamwork’ (n = 45), ‘leadership’ (n = 12)and ‘decision making’ (n = 20) lists were discussed by group 2.Group 3 discussed the ‘task management’ (n = 57), ‘managingstress’ (n = 34) and ‘coping with fatigue’ (n = 10) lists and group 4discussed the ‘situation awareness’ (n = 103) skill list. This distri-bution of the data meant that each group had approximately thesame number of items to discuss, in their respective 3-hour session(see Table 1, columns 1 and 3). Because of the limited time avail-able, the lists were each discussed by one focus group only.

Results

Following four focus group sessions, the preliminary SPLINTStaxonomy contained eight categories with 28 underlying elements.Columns 4 and 5 in Table 1 show the labels for the 28 elementsand example behaviours provided by the focus groups.

Refining the SPLINTS taxonomy

The aim of the second stage of the taxonomy development wasto refine the preliminary taxonomy. The resulting SPLINTS tax-onomy was to contain only the most critical non-technical skillsfor safe and effective scrub practitioner performance in order to beusable in the operating theatre setting.

Design

An expert review panel independently and collaboratively refinedthe preliminary skill taxonomy.

Participants

The review panel consisted of two psychologists (LM and RF) andan experienced theatre nurse on the project steering group (JM).

Materials

The lists of skills, elements and possible behavioural markersderived from the focus groups were produced as printouts forindependent and collective expert panel review. As the iterativeprocess progressed, refined printouts were produced for furtherindividual and collective revision and discussion by the reviewpanel.

Procedure

The review panel individually read the printouts of data then, withreference to the system design specifications, made notes regardingthe suitability and observability of the elements and behaviouralmarkers. The review panel met after their individual examinationsof the data, to begin the skill refinement process. This involvedremoving skill elements that were deemed unobservable, forexample, ‘developing knowledge of individual surgeons’ person-alities and preference’ [32]. Also, some elements were able to becombined, for example, timing when to interrupt and judging thecorrect time for equipment to be brought to the table were com-bined under the ‘anticipating’ element of situation awareness. Thisprocess of individually reviewing and then meeting to refine theskill set occurred on two further occasions so that the most criticalbehaviours (final judgement by JM) in as few skill categories andelements as possible were included in the taxonomy. After eachreview panel discussion, revised printouts of the potential skillelements and behavioural markers were produced. On the thirdmeeting, the prototype skill set was agreed.

Results

The iterative review process resulted in reducing the skill set fromeight categories, with 28 underlying elements to a concise skill setcontaining three categories each with three underlying elements.Table 3 details the three categories and nine elements in columns 1and 2, respectively. Column 3 lists the element labels previouslyprovided by the focus groups, and examples of behavioural markersfor good and poor performance are shown in columns 4 and 5.

Rating scale

In order for scrub practitioners to use the prototype taxonomy, arating scale was required. Studies of behavioural rating systemswith 5-point rating scales found that the point at the ‘outstanding’end of the scale was rarely used and carried an additional cost toconsensus agreement [33]. Pilot testing of the system for anaes-thetists (i.e. ANTS) found a 3-point scale restricting, resulting in a4-point scale being chosen for the final taxonomy [22]. A 4-pointrating scale was selected for the SPLINTS system, and labels anddescriptors were written for each of the four scale points (seeTable 4).

The labels for the 4-point scale were 1 = poor, 2 = marginal,3 = acceptable and 4 = good, since this avoided using terms such as‘substandard’ or ‘outstanding’ as had been used in the Gaba et al.[33] study. In addition to the 4-point scale, a rating option of ‘notrequired’ was included in the SPLINTS rating scale for occasionswhen a particular behaviour was not observed because it was notrequired by the scrub practitioner in that particular surgical case.

SPLINTS v1.0 handbook

Following the refinement process, the prototype SPLINTS tax-onomy had to be produced in a format suitable to be used in thenext stage of development, that is evaluating the SPLINTS system[34]. The SPLINTS handbook v1.0 was designed and printed withthe understanding that it could be adapted, if necessary, followingthe results of the evaluation study.



DiscussionNon-technical skills are important for safe and effective perfor-mance in high-risk industries [11] and methods for training and

assessing these skills for airline pilots [17] and individuals in theoperating theatre [18,19] have been developed. Since the finalstage in the development of these tools had not previously beenreported in detail, this paper sought to address that gap in theliterature. Having a framework for teaching and assessing thisaspect of performance would provide those engaged in nursingeducation with a structured means for providing formative assess-ment on performance of these skills. Milligan [35] previouslyargued the importance of adopting a human factors approach innursing education and the adaptation of existing non-technicalskills systems [22,27] have recently been used to introduce theterminology to final year nursing students when debriefing perfor-mance in ward simulation exercises [36].

During the development process, to ensure the inclusion of onlythe most essential non-technical skills, some skill categories weremerged together or were incorporated as elements of other skills.For example, ‘leadership’ was not included in the taxonomy at thecategory level; however, the behavioural marker ‘demonstratesleadership qualities when appropriate’ was included in the ‘actingassertively’ element of the ‘communication and teamwork’ cat-egory. This reduction process was necessary so that the system was

Table 3 Results of expert panel review process

Category Element Focus group labels* Good behavioural marker Poor behavioural marker

Situationawareness

Gatheringinformation

Listening (SA) • Watches surgical procedure • Fixates on one task• Collects information from other team

members• Fails to listen to instructionsWatching (SA)

Awareness (SA)

Recognizingandunderstandinginformation

Understanding (SA) • Switches between tasks efficiently • Does not prioritize tasks

• Responds late or not at all tochange in pace of procedure

• Provides correct instrument evenwhen not named/ incorrectly named

Atmosphere (SA)Knowledge (SA)

Anticipating Anticipating (SA) • Predicts when plan of procedure isgoing to change

• Loses track of surgical activity,that is, caught unawareTiming (SA, DM))

• Asks for items late• Times requests appropriately

Communicationandteamwork

Actingassertively

Assertiveness (C) • Gives clear instructions • Fails or slow to communicaterequirements• Demonstrates leadership qualities

Exchanginginformation

Verbal (C) • Provides team with information • Makes requests withoutspecifying for whomcommunication is intended

Non-verbal (C) • Uses non-verbal signals whereappropriateSharing (TW)

Co-ordinatingwith others

Clarifying (TW) • Suggests alternative options/equipment

• Fails to share information aboutevolving surgical planCo-ordinating (TW)

Leading (TW) • Supports others by providing helpand assistance

• Allows interruptions to disruptflow of procedureEquipment (DM, TM)

Taskmanagement

Planning andpreparing

Planning and preparing (TM) • Organizes tasks • Opens sterile equipment/supplies indiscriminatelyOrganizing (TM) • Utilizes time during breaks in

procedure for other preparatorytasks

Equipment (TM, DM) • Confuses order of tasks whichbest promote procedure flow

Providing andmaintainingstandards

Maintaining standards (TW) • Protects sterile field and instruments • Does not adhere to protocolsEquipment (TM, DM) • Does not display effective

organization of own workspace• Controls volume of music

Timing (DM, SA)Coping with

pressureCoping with pressure (MS) • Does not rise to others’ emotional

outbursts• Raises voice unnecessarily

Stress in self (MS) • ‘Freezes’ and unable to functioneffectivelyStress in others (MS) • Delegates tasks where possible to

ease pressure of situation

*Skill list given to focus group in brackets.C, communication; DM, decision making; MS, managing stress; SA situation awareness; TW, teamwork.

Table 4 SPLINTS rating options

Ratingpoint Label Description

1 Poor Performance was not acceptable and couldhave potentially endangered patient safety,remedial action is required

2 Marginal Performance indicated cause for concern,considerable improvement is needed

3 Acceptable Performance was of a satisfactory standardbut could be improved

4 Good Performance was of a consistently highstandard, enhancing patient safety; it couldbe used as a positive example for others

N/R Not required Not required; skill was not observed becauseit was not required in this case



as concise as possible so as to be usable for practitioners in theoperating theatre. Some skills had to be removed from the finalskill set. For example, ‘stress management’ is a topic thatis included in most Crew Resource Management training pro-grammes [37]; however, it is not necessarily included in behav-ioural rating systems. This is because of the difficulties associatedwith observing this skill in other people unless extreme symptomsare displayed, at which point it would be evident in performance ofother skills in the taxonomy [11]. Although ‘managing stress’ wasfelt by nursing experts to be a critical skill for scrub nurses toacquire, rather than being a skill category on its own, ‘coping withpressure’ was included as an element of the ‘task management’skill category.

‘Communication’ was a skill category that was not included inthe NOTECHS behavioural rating system for pilots as the pilotexperts developing that system believed that rather than being aseparate category, communication skills were inherent in all theother categories as the exemplar behaviours all involved commu-nication [17].

However, ‘communication’ was included in this prototypeSPLINTS system but was combined with ‘teamwork’ to form a‘communication and teamwork’ category. The experienced practi-tioners in the focus groups felt that good verbal and non-verbalcommunication were essential skills for scrub practitioners toacquire. ‘Assertiveness’ was included as an element under this skillcategory since it is viewed as a difficult, but necessary, skill forscrub nurses to acquire [38]. Since insufficient or ineffective com-munication between team members in the operating theatre settinghas been recognized as a contributing factor to some adverseevents [39], it could be argued that including ‘communication’ asa skill category for operating theatre staff is a positive step towardsexplicitly dealing with this safety critical aspect of non-technicalskill performance.

It was apparent that aspects surrounding ‘planning and prepa-ration’ were crucial to enabling safe and effective performanceduring the intraoperative phase of the surgical procedure. In theWorld Health Organization [40] surgical patient safety checklist,checking patient identities and patient positioning are activitiesthat should be performed prior to the intraoperative phase. It wasagreed that these are critical tasks that should be included inthe SPLINTS taxonomy because if these tasks are not completedcorrectly preoperatively resulting in the wrong patient being on thetable or the wrong side of the patient being prepped for surgery, itwould be of no consolation even if the non-technical skills wereperformed effectively during the operation.

The SPLINTS taxonomy was subsequently evaluated by theatrepractitioners (n = 34) in Scottish hospitals (n = 5) using standard-ized simulated operating theatre video scenarios to test the psy-chometric properties, including inter-rater reliability of the system.Feedback on the non-technical skills included in the taxonomy wasalso gathered from practitioners in that evaluation study [34]. Thenext step in the research project is to test the usability of the systemas a training and assessment tool in the real operating theatreenvironment. Having reliable tools to assess and measure non-technical skills performance of members of the operating theatreteam provides a common language for team members to discussand train this important aspect of performance. The SPLINTSsystem could prove a useful tool in the scrub practitioner’sarmoury against adverse events.

Strengths and limitations

The two-stage process of using focus groups with varying levels ofnon-technical skill and behavioural marker system knowledge, andthen an expert panel to review the list of skills appeared to be aneffective, if somewhat lengthy process. This method was opento possible bias arising from the different levels of experience inthe focus groups, and by the previous experiences of non-technicalskills taxonomy development of the psychologists in developingthe prototype taxonomy. Reviewing the data independently andthen collaboratively in the review panel minimized those biases.

ConclusionNon-technical skills combined with technical expertise are importfor safe and effective performance in the operating theatre [11] andthis aspect of performance is not currently explicitly taught orassessed for scrub practitioners. This paper described one keystage of the development process of the SPLINTS system v1.0 forscrub practitioners’ non-technical skills. This prototype behav-ioural rating system was produced as a handbook for users (avail-able at http://www.abdn.ac.uk/iprc/splints).

AcknowledgementsThis study was funded by a Scottish Funding Council grantawarded to the Scottish Patient Safety Research Network. Wewould also like to thank the theatre nurse managers who partici-pated in the study.

References1. Flin, R. & Mitchell, L. (eds) (2009) Safer Surgery: Analysing Behav-

iour in the Operating Theatre. Farnham: Ashgate.2. Vincent, C., Aylin, P., Franklin, B. D., Holmes, A., Iskander, S.,

Jacklin, A., Moorthy, K. (2008) Is health care getting safer? BritishMedical Journal, 322, 517–519.

3. Kohn, K. T., Corrigan, J. M. & Donaldson, M. S. (eds) (1999) To Erris Human: Building a Safer Health System. Washington, DC: NationalAcademy Press.

4. Brennan, T. A., Leape, L. L., Laird, N. M., Herbert, L., Localio, A. R.,Lawthers, A. G., Newhouse, J. P., Weiler, P. C., Hiatt, H. H. (1991)Incidence of adverse events and negligence in hospitalised patients.Results of the Harvard Medical practice study I. The New EnglandJournal of Medicine, 324, 370–376.

5. deVries, E. N., Ramrattan, M. A., Smorenburg, S. M., Gouma, D. J. &Boermeester, M. A. (2008) The incidence and nature of in-hospitaladverse events: a systematic review. Quality and Safety in HealthCare, 19, 216–223.

6. Taylor, M. & Campbell, C. (2000) The multi-disciplinary team in theoperating department. In Back to Basics: Perioperative Practice Prin-ciples (ed. D. Plowes), pp. 44–48. Harrogate: National Association ofTheatre Nurses.

7. Smith, C. D. (2011) The retained surgical specimen, an unappreciatedretained foreign object. Journal of Laparoendoscopic and AdvancedSurgical Techniques, 21 (8), 737–739.

8. Gawande, A. A., Studdert, D. M., Orav, E. J., Brennan, T. A. & Zinner,M. J. (2003) Risk factors for retained instruments and sponges aftersurgery. New England Journal of Medicine, 348, 229–235.

9. Altman, D. G. (1991) Practical Statistics for Medical Research.London: Chapman and Hall.



10. Beesley, J. & Pirie, S. (eds) (2005) Standards and Recommendationsfor Safe Perioperative Practice. Harrogate: National Association ofTheatre Nurses.

11. Flin, R., O’Connor, P. & Crichton, M. (2008) Safety at the Sharp End.A Guide to Non-Technical Skills. Aldershot: Ashgate.

12. Flin, R. & Patey, R. (2009) Improving patient safety through training innon-technical skills. British Medical Journal, [Editorial]. 339, 3595.

13. Gawande, A. A., Zinner, M. J., Studdert, D. M. & Brennan, T. A.(2003) Analysis of errors reported by surgeons at three teaching hos-pitals. Surgery, 133, 614–621.

14. Lingard, L., Espin, S., Rubin, B., Whyte, S., Colmenares, M. & Baker,G. R. (2005) Getting teams to talk: development and pilot implemen-tation of a checklist to promote interprofessional communication inthe OR. Quality and Safety in Healthcare, 14, 340–346.

15. Catchpole, K. R., Mishra, A., Handa, A. & McCulloch, P. (2008)Teamwork and error in the operating room: analysis of skills and roles.Annals of Surgery, 247, 699–706.

16. Sexton, J. B., Thomas, E. J. & Helmreich, R. L. (2000) Error, stress,and teamwork in medicine and aviation: cross sectional surveys.British Medical Journal, 320, 745–749.

17. Flin, R., Martin, L., Goeters, K., Hörmann, H., Amalberti, R., Valot, C.& Nijhuis, H. (2003) Development of the NOTECHS (non-technicalskills system) for assessing pilots’ CRM skills. Human Factors andAerospace Safety, 3, 95–117.

18. Fletcher, G., Flin, R., McGeorge, P., Glavin, R., Maran, N. & Patey, R.(2003) Anaesthetists’ Non-Technical Skills (ANTS): evaluation of abehavioural marker system. British Journal of Anaesthesia, 90, 580–588.

19. Yule, S., Flin, R., Paterson-Brown, S., Maran, N., Rowley, D. R. &Youngson, G. G. (2008) Surgeons’ non-technical skills in the operat-ing room: reliability testing of the NOTSS behaviour rating system.World Journal of Surgery, manuscript under review; 32, 548–556.

20. Crossley, J., Marriott, J., Purdie, H. & Beard, J. D. (2011) Prospectiveobservational study to evaluate NOTSS (non-technical skills for sur-geons) for assessing trainees’ performance in the operating theatre.British Journal of Surgery, 98, 1010–1020.

21. Graham, J., Giles, E. & Hocking, G. (2007) Using ANTS for work-place assessment. In Safer Surgery: Analysing Behaviour in the Oper-ating Theatre (eds R. Flin & L. Mitchell), pp. 189–201. Farnham:Ashgate.

22. Fletcher, G., Flin, R., McGeorge, P., Glavin, R., Maran, N. & Patey, R.(2004) Rating non-technical skills: developing a behavioural markersystem for use in anaesthesia. Cognition Technology and Work, 6,165–171.

23. Yule, S., Flin, R., Maran, N., Rowley, D., Youngson, G. G., Duncan, J.& Paterson-Brown, S. (2009) Development and evaluation of theNOTSS behaviour rating system for intraoperative surgery (2003–2008). In Safer Surgery: Analysing Behaviour in the OperatingTheatre (eds R. Flin & L. Mitchell), pp. 7–25. Farnham: Ashgate.

24. Mitchell, L. & Flin, R. (2008) Non-technical skills of the operatingtheatre scrub nurse: literature review. Journal of Advanced Nursing,63, 15–24.

25. Mitchell, L., Flin, R., Yule, S., Mitchell, J., Coutts, K. & Youngson,G. G. (2011) Thinking ahead of the surgeon: an interview study toidentify scrub nurses’ non-technical skills. International Journal ofNursing Studies, 48, 818–828.

26. van Avermaete, J. A. G. & Kruijsen, E. (eds) (1998) The Evaluationof Non-Technical Skills of Multi-Pilot Aircrew in Relation to theJAR-FCL Requirements: Final Report NLR-CR-98443. Amsterdam:National Aerospace Laboratory (NLR).

27. Yule, S., Flin, R., Paterson-Brown, S., Maran, N. & Rowley, D. (2006)Development of a rating system for surgeons’ non-technical skills.Medical Education, 40, 1098–1104.

28. Whiddett, S. & Hollyforde, S. (2006) A Practical Guide to Competen-cies: How to Enhance Individual and Organisational Performance, 2ndedn. London: Chartered Institute of Personnel and Development.

29. Krueger, R. A. (1994) Focus Groups: A Practical Guide for AppliedResearch. Thousand Oaks, CA: Sage.

30. Barbour, R. S. & Kitzinger, J. (eds) (1999) Developing FocusGroup Research. Politics, Theory and Practice. London: SagePublications.

31. Gordon, S. E. (1994) Systematic Training Program Design: Maximiz-ing Effectiveness and Minimizing Liability. Englewood Cliffs, NJ:Prentice-Hall, Inc.

32. Riley, R. & Manias, E. (2006) Governance in operating room nursing:nurses’ knowledge of individual surgeons. Social Science and Medi-cine, 62, 1541–1551.

33. Gaba, D. M., Howard, S. K., Flanagan, B., Smith, B. E., Fish, K. J. &Botney, R. (1998) Assessment of clinical performance during simu-lated crises using both technical and behavioral ratings. Anesthesiol-ogy, 89, 8–18.

34. Mitchell, L., Flin, R., Yule, S., Coutts, K., Mitchell, J. & Youngson,G. G. (2011) Evaluation of the SPLINTS system for scrub practitio-ners’ non-technical skills. International Journal of Nursing Studiesin.Available at: http://dx.doi.org/10.1016/j.ijnurstu.2011.08.012 (lastaccessed 18 November 2010).

35. Milligan, F. J. (2007) Establishing a culture for patient safety – the roleof education. Nurse Education Today, 27, 95–102.

36. Pearson, E. & McLafferty, I. (2011) The use of simulation as a learn-ing approach to non-technical skills awareness in final year studentnurses. Nurse Education in Practice, 11 (6), 399–405.

37. Kanki, B., Helmreich, R. L. & Anca, J. (eds) (2010) Crew ResourceManagement, 2nd edn. San Diego, CA: Elsevier.

38. Bromiley, M. & Mitchell, L. (2009) Would you speak up if theconsultant got it wrong? . . . and would you listen if someone saidyou’d got it wrong? Journal of Perioperative Practice, 19, 326–329.

39. Helmreich, R. L. & Schaefer, H. G. (1994) Team performance inthe operating Room. In Human Error in Medicine (ed. M. S. Bogner),pp. 225–253. Hillsdale, NJ: Lawrence Erlbaum Associates.

40. World Health Organization (WHO) (2009) Safe surgery saveslives: the second global patient safety challenge. Available at: http://www.who.int/patientsafety/safesurgery/en/ (last accessed 28 May2011).



development of a behavioural marker system for scrub practitioners' non-technical skills...

Documents