failure modes and effects analysis (fmea) for partial ...€¦ · failure modes and effects...
TRANSCRIPT
Failure Modes and Effects Analysis (FMEA) for Accelerated Partial Breast Irradiation via High Dose‐Rate Intracavitary Brachytherapy
Jeffrey F. Williamson1, Bruce R. Thomadsen2 , Geoffrey S. Ibbott 3, and Sasa Mutic 4
1Virginia Commonwealth University, 2University of Wisconsin; 3MD Anderson Cancer Center; 4Washington
University
Disclosures and Acknowledgments
• Disclosures:– JFW has research/service contracts with Varian Medical Systems, NIH, and AAPM
– JFW has never led formulation of FMEA in real‐life clinical application
• Acknowledgments– Chapter coauthors: Ibbott, Thomadsen, and Mutic– Clinical input and advice from Douglas Arthur, M.D.– Statistics expertise from Nitai Mukhopadhyay, Ph.D.– Slides from Saiful Huq, Ph.D.
Learning Objectives• Illustrate the application of TG‐100 Risk analysis techniques to balloon‐applicator HDR breast brachytherapy
• Review failure modes rated to be highest risk• Review QA/QC strategies for mitigating high risk failures
4
Example: Risk-Informed QM Formulation for Brachytherapy
• Scenario: Accelerated partial breast irradiation using multi-catheter balloon HDR brachytherapy applicator
– CT-based evaluation and planning– Multicatheter balloon applicator, e.g., Contura– Automated plan transfer but not full EMR charting
• History of this project– 2005: TG-100 assigns 4 coauthors to devise a
brachytherapy risk analysis case study– 2008: dropped from TG-100 report– 2010 - 2013: JFW revives project. FMs and scenarios
revised and extended with feedback from BRT.– 2013: Chapter authors ranked revised FMs.
5
Image-Guided Balloon Catheter PlacementAccelerated Partial Breast: MammoSite HDR BTx
• gg
Intraoperative Ultrasound
Visualize LumpectomyCavity: Select Approach
Assess conformality
Intraop/PostOp CT
Assess conformality
Assess SymmetryD. Arthur, VCU
Assess Skin Distance
6
Multi-Cath Balloon Applicator Mismatch errors
• ee
Asymmetric loading to spare skin
and chestwall
TG-100 Risk Analysis Steps• Steps
1. Define process by creating a process map2. Failure modes and effects analysis (FMEA): Identify
threats to success (failure modes) and rank according to risk
3. Fault-tree Analysis (FTA): Propagation of failures through system and placement of QM interventions
4. Develop QA or QC interventions to mitigate risk
7
TG-100 Risk Analysis Steps• Process Map: Step 1
– Delineate and then understand the steps in the process to be evaluated
– Visual illustration of the physical and temporal relationships between the different steps of a process
– Demonstrates the flow of these steps from process start to end
• prospective risk analysis for hypothetical clinical process modeled on VCU and UW-Madison processes
– Assumes NO QA or QC checks– Partial automation of EMR and data transfer
8
9
Breast Brachytherapy Process Map
TG-100 Risk AnalysisStep 2 FMEA
• Step 2a: For each process step, ask the following questions
– What could possibly go wrong ? (enumerate/ describe failure modes)
– How could that happen? (what are possible causes of FM?)
– What effect would such an undetected failure have? (Potential impact on quality)
• Step 2b: Assess risk of FM by estimating O, S, and P
• Present analysis: 96 Failure Modes 10
Process tree
Sub-process #1
Sub-process #17
Sub-process #19
Step #4Step #1 Step #j
Failure mode #2Failure mode #1 Failure mode #k
Causes of failure #6Causes of failure #1 Causes of failure #m
Effects of failure #4Effects of failure #1 Effects of failure #n
Step 2a: enumerate FMEA Failure Modes
12
Assess Risk Posed by Each FMStep 2b
• For each subprocess, enumerate the possible scenarios, i.e., Failure Modes (FM), that could lead an unsuccessful treatment: 96 FMs
– Identify causes and effect on process outcome• Assess risk to successful outcome posed by
each FM assuming no QA
– Assign O,S, and P a value from 1-10– 4 Observers: Ibbott, Mutic, Williamson, Thomadsen– Significant additions/modifications by JFW
• Reorder list in terms of descending RPN
Likelihood of Severity of Likelihood ErrorRisk occurrence consequences Detected
O S P Risk Proba
No
bility Number R S P
t
PN O
TG-100 FMEA Rating Scales
• ff
13
14
Combined
ItemNumber Major Processes Step Potential Failure Modes
Potential Causes of Failure JFW Comments and descriptive scenario
Potential Effects of Failure
AVG O AVG S AVG D Avg RPN
1 Imaging and diagnosis
RO reviews EMR prior to RO consult
Med Onc or Surgeon consultation mis interprets or mis represents primary clinical findings ( imaging studies, path reports, etc); incorrectly stages patient, and recommends BCT and APBI for patient that is not appropriate candidate
RO bases Tx recommendation on secondary MD report rather than reviewing primary clinical findings
and discovering the ups tream error
Upstream phys ician error potentially discoverable by Rad
Onc s ince primary c linical data is available We should recommend that the RO performs their dut ies
diligently .
wrong/very wrong dose distribut ion 5.00 8.25 5.50 269.3
2 Imaging and diagnosis
RO reviews EMR prior to RO consult
path or biomarker reports is incorrect due to mis labeling of surgical spec imen or biomarker report . Hence pat ient is unders taged and inappropriately offered BCS by Med Onc and Surgeon
RO recommendation for APBI is fully consistent with prior EMR
An error not eas ily discoverable by Rad Onc Based on the worse case.
Very wrong dose 4.25 8.75 8.25 309.5
3Patient database
information
Entry of patient data in RO EMR or writ ten
chart
Incorrect patient ID data Documentation error
Wrong patient ID leading misfiling of demographic and c linical data from hospital DB; identification of wrong patient
Very wrong dose 3.00 8.75 2.75 70.0
4 Patient Database Information
Entry of patient data in RO EMR or writ ten
chart
Correct patient ID data but clinical
findings/ images from wrong pat ient loaded
into RO EMR
Omission in entry, incomplete pat ient history
Incorrect clinical findings leads to faulty decision to t reat or downstream peer-review correc tion
Very wrong dose 5.00 7.75 3.75 154.5
5 Consultat ion and decision to t reat
Decision of treatment technique and protocol
Clinically inappropriate patient selected for
APBI
misinterpretating of clinical findings incomplete H&P
Even though upstream clinical data are correct, Error by RO assessing indicat ions and contraindications to APBI., e.g. , SLN+ with Surg untreated axilla. RO mis represents or neglects key finding and offers inappropriate treatment plan to patient
Very wrong dose 4.25 7.75 7.75 252.8
6Consultation and
dec ision to treat or imaging/diagnos is
Decision of treatment technique and protocol or imaging/diagnosis
patient with radiographically too
large or closed seroma cavity selected
RO error in interpreting imaging studies; inappropriate imaging used; or poor imaging quality
JFW: New failure mode
Very wrong dose if not detec ted; more likely
major inconvenience or infection from
unnecessary invas ive procedure
4.75 6.25 4.75 140.3
7 Consultat ion and decision to t reat
Decision of treatment technique and protocol
Clinically inappropriate patient selected for
APBI
Upstream error in document ing clinical findings . Various scenarios with differing levels of discoverability
Because of upstream Imaging and Diagnosis errors, RO offers inappropriate treatment plan to the patient
Very wrong dose 4.00 7.50 5.50 180.0
First 7 Failure Modes
15
8 Highest Risk FMs
• Red entries: New JFW FMs or modified RPN
16• Red entries: New JFW FMs or modified RPN
e r M ajor Processes Step Pote ntial Fa ilure Modes
Potential Causes of Failure JF W Comments and descriptive scenario
Potential Effects o f Failu re
AVG O AVG S AVG D Avg RPN
Init ial treatment Run treatment Incorrec t balloon rott ion
Inattent ion,.miss ing required check , poor documentation
Failure to check that applicator rotat ionmatches plan, so that wrong dwell posit ion sequence is opposing sk in bridge.
Wrong dose distribut ion
5.25 7.25 7.25 302.0
Plan approval MD reviews planPlan fails to sat isfy
c linical intentions , bad plan approved
Inadequate or incomplete review, failure to reoptimize inadequate plan, miscom munication, inat tent ion, lack of s tandardized procedures
Error is that MD approves a plan that violates cons traints same phys ician previously approved and subopt imal plan is loaded into EMR in prep for treatment
Subopt imal plan, poor treatment used
5.75 5.75 8.25 299.0
Treatment Planning Optim izat ion sett ings
Specification of optimization m ethod, dose-point locations,
prescribed dose protocol not followed
accurately
Inadequately trained personnel, inat tent ion, poor inter-disc iplinary comm unicat ion
Planners following different protocol for sett ing constraints and goals
Very wrong dose 5.25 8.00 6.75 293.0
Treatment planning Dwell pos ition cons truc t ion
Distal-mos t dwell location inaccurately
digit ized
Inadequately trained, inattent ion, poor inter-disciplinary comm unicat ion
E,g,, offset from tip to dwell 1 used as s tandard pract ice in c linic not known
Wrong dose distribut ion
4.50 7.25 7.75 287.5
Treatment planningCatheter
localizat ion/labeling: s ingle catheter
Catheter t rajectory inaccurately localized
W rong catheter s lice images, inadequately trained personnel, poor inter-disciplinary comm unicat ion, inattent ion
Many possible scenarios : operator ignorance of (a) which vis ible s truc ture denotes inner catheter tip, (b) offset from dis talmos t source center and catheter tip. Also s loppy contouring or import ing wrong plan template
Wrong dose distribut ion
4.25 8.00 7.75 285.5
Treatment planning Dwell pos ition cons truc t ion
Random error: Treatment length incorrect (wrong
transfer tube length, wrong sounding
information, wrong dwell spac ing)
Inadequately trained personnel, inat tent ion, poor inter-disc iplinary comm unicat ion, default distances, equipment failure
Mult iple error pathways: (1) Inaccurate sounding measurements from pos t-op imaging; (2) wrong spac ing and/or offsets; (3) afterloader spec ific calibrat ion correc tions , e.g. , Varian QuickConnect 14 mm correct ion, ignored; (4) arithmetic error
Very wrong/wrong dose dis tribution
5.00 8.50 6.25 284.0
Next 6 Highest Risk FMs
Fault Tree Analysis and Designing QM interventionsSteps 3 and 4
• Step 3: Create Fault Trees (optional)– Time consuming: Limit FTA to selected FMs– Visualize interactions between FMs possibly in different
process tree branches– JFW: helped me refine list of FMs and scenarios
• Step 4: Design QM intervention– Rank FMs according decreasing risk and severity– Mark high RPN/S FMs on fault and process trees– FTA guides optimal placement of intervention– Design intervention: balance cost, specificity,
sensitivity and benefit
17
Fault Tree AnalysisStep 3: TG100 risk analysis methodology
• FTA compliments process tree• Leftmost box is the failure (error)
• Each daughter node is a FM that could cause the error
• Works backwards in time (to the right) until root cause is reached
• Models propagation of error through system
18
• ‘OR’ means error occurs if any one of antecedent FMs occurs
• ‘AND’ means all antecedent FM’s must be realized for error to occur
• No QA/QC assumed
• Relevant FMs scattered across at least 4 PT branches
• Interactions
19
Program treatment unit
failure
Wrong data file imported
Inconsistency between treatment
program and default operating
parameters
Software failure
InattentionOr
Or
Dose in wrong location due to source position
Units length distance is incorrect
Unit step size inaccurate
Applicator in wrong location
Or
Single or multiple catheter failure
Catheter trajectory inaccurately
localized
Incorrect catheter number asssigned
Wrong catheter slice images
Inadequately trained personal
Poor labeling on photographs
Poor image quality
OrOr
Or
Dwell position construction
failure
Distal-most dwell location
inaccurately digitized
Treatment length incorrect (wrong
transfer tube length, wrong
sounding information, wrong
dwell spacing)
Inadequately trained personal
Poor images
Default distances used
Equipment failure
Or
Or
Or
48
Or
Systematic offset Commissioning
failure
Catheter localization
failure
Wrong catheter position Marked
Catheter indicators not inserted fully
Or
Or
Non-Positional Failure Modes
Post-procedure CT imaging
error
Sounding measurement error
Channel numbering error: marking or
recording
Channel and applicator numbers not matching when connecting transfer tubes to applicator
Wrong length transfer tube
OrConnect transfer
tubes to applicator failure
Non-Positional Failure Modes
Poor quality/incomplete
images
Initial treatment failure
Or
Treatment planning
Error
Channel Mismatch
Or
Non-Positional Failure Modes
Incorrect Catheter Polar rotation
26
25
26
25
23,24
51
49
50
46
52
52 53
73
75
72, 70, 71
86
76
Source Positioning ErrorFault Tree
• Error types– Channel mismatch– Incorrect Tx length– Incorrect step length
• Top level causes– Post procedure
imaging error– Tx Planning error– Error in treatment
setup or device programming
20
Dose in wrong location due to source position
Units length distance is incorrect
Unit step size inaccurate
Applicator in wrong location
Or
Single or multiplcatheter failure
Dwell positionconstruction
failure
O
Or
Non-Positional Failure Modes
Post-procedure CT imaging
error
Initial treatment failure
Treatment planning
Error
Channel Mismatch
Or
Source Positioning ErrorFault Tree
• Incorrect information /poor images Dwell position programming error
– Channel numbering or documentation– Catheter length measurement– Imaging performed with incorrect marker position
• QM interventions– QC: second therapist assists with measurements– QA: Independent check of localization data before patient leaves
imaging suite
21
Catheter localization
failure
Wrong catheter position Marked
Catheter indicators not inserted fully
Or
Or
Non-Positional Failure Modes
Post-procedure CT imaging
error
Sounding measurement error
Channel numbering error: marking or
recording
Poor quality/incomplete
images
And Physicist/dosimetrist
check images failure
Adequate QM program for planning and afterloader
systems
And
Assisting therapist misses errors
26
25
26
25
23,24
Post-procedure CT imaging Localization Errors
22
Post-Procedure ImagingLocalization steps
23
Localization FMs:Treatment Planning
• Catheter trajectory delineation error– Dwell 1 length error
» Systematic positional offset error– Dwell position digitization error
• channel mismatch error
Single or multiple catheter failure
Catheter trajectory inaccurately
localized
Incorrect catheter number asssigned
Wrong catheter slice images
Inadequately trained personal
Poor labeling on photographs
Poor image quality
Or Or
Or
Dwell position construction
failure
Distal-most dwell location
inaccurately digitized
Treatment length incorrect (wrong
transfer tube length, wrong
sounding information, wrong
dwell spacing)
Inadequately trained personal
Poor images
Default distances used
Equipment failure
Or
Or
Or
48
Or
Systematic offset Commissioning
failure
Wrong catheter position Marked
Catheter indicators not inserted fully
Or
Or
Non-Positional Failure Modes
Sounding measurement error
Poor quality/incomplete
images
Initial treatment Operator check
And
Physicist check plan failure
Adequate QM program for planning and afterloader
systems
Treatment planning
Error
26
25
26
23,24
51
49
50
46
52
52 53
Treatment length
Dwell positions
24
Example: Systematic Offset Error• Systematic source
positioning error: caused by invalid treatment length estimation protocol
• Varian “quick connect” indexer interface
– 14 mm offset compared to standard transfer tube connector with usual transfer tube-applicator combination length measurement
– No Software offset or hardware interlock initially provided
25
TG-56 Structured HDR Positional Accuracy Tests
dwell 1 (1500 mm) position
indexer reference L1 = 0.0
transfer tube
Tube Length: L (1218 mm)
t
Fully Inserted Radiographic Marker
LRM
transfer tube L1 (programmed length) = 1500
mm
Tube Length: L (1218 mm)
t
Radioactive Source Programmed to 1500 mm
PDR Head
Compare for
Coincidence
0"Odcalibrated dummy ribbon
d : Dummy Insertion Depth
iApplicator Orfice
26
Mitigating RTP Localization FMs
• Implement well-defined, rigidly followed procedures: – Adequate patient volume
• QC: use only one transfer tube length & use equi-length catheters
• QA: Final physics plan review focus on dwell position
Single or multiple catheter failure
Catheter trajectory inaccurately
localized
Incorrect catheter number asssigned
Wrong catheter slice images
Inadequately trained personal
Poor labeling on photographs
Poor image quality
Or Or
Or
Dwell position construction
failure
Distal-most dwell location
inaccurately digitized
Treatment length incorrect (wrong
transfer tube length, wrong
sounding information, wrong
dwell spacing)
Inadequately trained personal
Poor images
Default distances used
Equipment failure
Or
Or
Or
48
Or
Systematic offset Commissioning
failure
Wrong catheter position Marked
Catheter indicators not inserted fully
Or
Or
Non-Positional Failure Modes
Sounding measurement error
Poor quality/incomplete
images
Initial treatment Operator check
And
Physicist check plan failure
Adequate QM program for planning and afterloader
systems
Treatment planning
Error
26
25
26
23,24
51
49
50
46
52
52 53
• Adequate device QA: – Maintain image quality– Eliminate offsets and
incorrect default parameters
– Consistency of procedure with device function
• QA/QC in red• Adequate device
QA protocol• Written procedures
– Redundancy– Uniformity– Patient volume
• Physics Checks – Simulation– Tx plan– Setup/RAL
programming
27
Program treatment unit
failure
Wrong data file imported
Inconsistency between treatment
program and default operating
parameters
Software failure
InnatentionOr
Or
Units length distance is incorrect
Unit step size inaccurate
Applicator in wrong location
Or
Single or multiple catheter failure
Catheter trajectory inaccurately
localized
Incorrect catheter number asssigned
Wrong catheter slice images
Inadequately trained personal
Poor labeling on photographs
Poor image quality
Or Or
Or
Dwell position construction
failure
Distal-most dwell location
inaccurately digitized
Treatment length incorrect (wrong
transfer tube length, wrong
sounding information, wrong
dwell spacing)
Inadequately trained personal
Poor images
Default distances used
Equipment failure
Or
Or
Or
48
Or
Systematic offset Commissioning
failure
Catheter localization
failure
Wrong catheter position Marked
Catheter indicators not inserted fully
Or
Or
Non-Positional Failure Modes
Post-procedure CT imaging
error
Sounding measurement error
Channel numbering error: marking or
recording
Channel and applicator numbers not matching when connecting transfer tubes to applicator
Wrong length transfer tube
OrConnect transfer
tubes to applicator failure
Non-Positional Failure Modes
Poor quality/incomplete
images
Initial treatment failure
Or
And
Operator check failure: imported parameters vs.
plan
And
Physicist check plan failure
And Physicist/dosimetrist
check images failure
Adequate QM program for planning and afterloader
systems
Treatment planning
Error
Failure: Physicist & MD
final setup check
And
Channel Mismatch
Or
Non-Positional Failure Modes
And
Assisting therapist misses errors
Incorrect Catheter Polar rotation
26
25
26
25
23,24
51
49
50
46
52
52 53
73
75
72, 70, 71
86
76
Mitigating Source Positioning Errors
28
Patient Selection Error FTA
with QA
– Upstream medic error in EMR “Imaging-Dx”
» Med or surg Onc error: recommends APBI
» Upstream histopath /biomarker error
• Two major FMs– Rad Onc misses
or ignores API clinical or technical contraindication
Are Pathology Errors Common?• ASTRO guidelines for APBI
– T1 uni-focal/centric GTV < 2 cm with margins > 2 mm– Negative SLN biopsy or ALND– Favorable histology/biomarkers
• Pignol: outside specimens of 77 patients referred for APBI reviewed inhouse
• 18.6% of patients deemed suitable for APBI on initial path, reclassified as “unsuitable”.
29Pignol, Clin Oncol 2012
30
Mitigating Patient Selection Errors• QM interventions and process redesign
– QC checklist to guide MD consultation– QA: Peer- and technical-review of MD decisions– QC: National electronic medical record
• Second review of pathology and biomarkers
31
TG-100 Site-Specific Treatment Schemas• Clinical parameters
– Pre-Tx eligibility criteria (Site, stage, histology, etc.) for this care pathway
– Overall plan, including sequencing of chemo, surgery, external and brachytherapy courses
– Pre-Tx Simulation and clinical instructions, e.g., fiducial marker placement, MR or PET imaging, dental extractions
• Planning parameters– GTV, CTV, normal tissue delineation guidelines;
image selection and registration– Prescribed dose, fractionation, IMRT class
solution, planning goals, and constraints. – Motion and uncertainty management techniques
32
Conclusions-I: Major lessons• FTA/FMEA is not rocket science: reinforces
decade-old TG-59 philosophy– Adequate device QC is essential to avoid systematic
errors– Planning & delivery processes must have built-in
catastrophic error detection and correction mechanisms– QA/QC must emphasize
» Redundancy (QC) when capturing key data» Compensating for weaknesses of devices, i.e., key actions where
interlocks don’t “force” desired outcomes
• QM of physician decision making is essential– Upstream histopath/biomarker error not uncommon– Fundamental departure from “MD command/control”
model: ancillary staff must be trained and empowered to question MD actions
33
Conclusions-II• Process mapping and FMEA advantages
– Focuses attention on process as well as device failures– Provides a vehicle for team to work collaboratively to
» better understand the process » Appreciate each other’s vulnerabilities » buy into core QM/QI values
– Most expert member gets to fix FM– Promotes clinical process uniformity so that desired
process-step outcomes get internalized– Better understanding of device-process interactions
helps physicist prioritize device QA• Downsides
– Resource intensive to build/use FMEA expertise– Not a mechanical, one-size-fits-all prescriptive
approach: requires judgment and individualization