Current Trends for Quality Assurance in Radiotherapy
Tommy Knöös, Sweden
Current Trends for Quality Assurance in Radiotherapy
Tommy Knöös, Sweden
3
Objectives
Can we improve the efficency of our quality assurance?
Quality assurance efforts are increasing with new modalites and technology
Can we change our approach?
Can we learn from other organisations?
2010-05-27 Dublin
Dublin2010-05-27
Page 4
Background The scientific world is full of
• Suspension/Tolerance levels• Action levels• Time intervals
For testing a huge amount of parameters• Treatment units• CT scanners• Treatment planning systems• Especially technical and physical ones
Focus of current QM/QA guidelines• measure functional performances of
radiotherapy equipment by measurable parameters
• set tolerances at desirable but achievable values
What is on a cancer patient’s mind?
• Safety
• Radiation therapy accident article published in NY Times
AAPM Summer School 20115Saiful Huq - New Paradigms for QM in RT
Dublin
6
How to avoid these?
2010-05-27
Lisa Norris
Brian Lara Cancer Centre Trinidad and Tobago
Can current QM standards prevent such events and meet the safety standards of
new clinical challenges?
Failure of QM:
• Insignificant errors in dose to the target • Major events that can injure or kill patients
Main goal of QM:
• Patients will receive the prescribed treatment correctly
AAPM Summer School 20117Saiful Huq - New Paradigms for QM in RT
Examples of current QA paradigms: Annual calibration of linac
that is it could have been spent checking things with a higher likelihood of failure
• The annual QA takes several days to complete
• If everything checks out, the effort was mostly wasted
• If some problem was found, how long had it been wrong?
• If the problem was significant then a daily check should be in place to prevent it
AAPM Summer School 20118Saiful Huq - New Paradigms for QM in RT
• We measure fluence maps for IMRT QA• This is certainly something that we would want to
have correct• However, if the system has been commissioned, and
then used correctly, why would this be wrong for the patient?
• If you worry about the leaf movement, even if the fluence map indicates correct movement for the first day of treatment, why do you think it might be correct in a week?
• Are there problems found this way? Almost never• Do we need QA for leaf movement, maybe daily???Saiful Huq - New Paradigms for QM in RT 9
AAPM Summer School 2011
Xample: IMRT QC
Problems with current QA paradigm
• These examples suggest that QA as performed currently might not be the optimal use of a physicist’s time
• Just performing all the recommended QA might not provide protection against events
Saiful Huq - New Paradigms for QM in RT 10AAPM Summer School 2011
Implementation of industrial tools
Dublin2010-05-27
Page 12
Quality management tools
Process mapping• Processes and their sub processes
Identify the processes which are more error prone• FMEA (Failure Mode and Error Analysis)• RCA (Root Cause Analysis)
Identify measures for control• SPC (Statistical Process Control)
Measure the quality of the measures/control points• Score actual and potential deviations (e.g.
ROSIS)
Dublin2010-05-27
Page 13
Quality management tools
Process mapping• Processes and their sub processes
Identify the processes which are more error prone• FMEA (Failure Mode and Error Analysis)• RCA (Root Cause Analysis)
Identify measures for control• SPC (Statistical Process Control)
Measure the quality of the measures/control points• Score actual and potential deviations (e.g.
ROSIS)
ICARO 20092009-04-28
Page 14
Process maps Process map is a visual demonstration of work processes The map shows how inputs, outputs, and tasks are linked together
• Identifies the major steps in the process• Who performs the steps
Provides a visual picture of the process Identifies and understand process deviations and vulnerabilities,
breakdowns, mistakes, delays - detects where improvements may be made
A.k.a flow charting Numerous formats or approaches exist - As-is and/or To-be
Imaging Volumes Planning Review
Prescription
Trearment (1-n)
Accelerator
Treatment finished
Treatment starts
TPSCT, PET/CT,
MR ...
ICARO 20092009-04-28
Page 15
Work plan
1. Establish process boundaries
2. Develop the data gathering plan
3. Interview the process participants
4. Generate the process map
5. Analyze and use the map
ICARO 20092009-04-28
Page 16
From Ford et al IJROBP, Vol. 74, No. 3, pp. 852–858, 2009
Dublin
17
2010-05-27
Dublin
18
Process mapping
2010-05-27
S Huq et al. (Red Journal QA issue)
ICARO 20092009-04-28
Page 19
Example of “swim lane” flow chart: Patient to have an diagnostic x-ray
From “Japanese Association of Healthcare Information Systems Industry“
Dublin2010-05-27
Page 20
Quality management tools
Process mapping• Processes and their sub processes
Identify the processes which are more error prone• FMEA (Failure Mode and Error Analysis)• RCA (Root Cause Analysis)
Identify measures for control• SPC (Statistical Process Control)
Measure the quality of the measures/control points• Score actual and potential deviations (e.g.
ROSIS)
Use process maps to identify critical steps in the process and/or when investigating incidents and
accidents
ICARO 20092009-04-28
Page 22
Identifying hazards and error prone sub-processes
FMEA – Failure Mode and Error Analysis During the FMEA, which is a proactive method, the
following questions have to be answered for each step in each sub-process; • a) what could possibly go wrong (potential failure mode), • b) how could that happen (i.e., what are the causes that
result in a failure mode) and finally • c) what effects would this failure mode produce (potential
effects of failure) Suitable for creating Fault Trees
ICARO 20092009-04-28
Page 23
FMEA
For each failure mode, estimate:• The probability the failure occurs (“O”)• The severity of it’s effects (“S”)• The probability that the failure will be
undetected (“D”) Form the “Risk Priority Number” RPN:
• RPN=O*S*D O,S,D range [1,10] Concentrate on the highest RPN
Step Potential failure modes
Potential causes
of failure
Potential
effects of
failure
O S D RPN
Comment
FMEAFor a given sub-process:
RPN = O x S x D [ 1 ≤ RPN ≤ 1000 ]
AAPM Summer School 2011Saiful Huq - New Paradigms for QM in RT 24
O, S, and D values used in FMEARank Occurrence (O) Severity (S) Detectability
(D)Qualitative Frequency Qualitative Categorization Estimated
Probability of going
undetected in %
1 Failure unlikely
1/10,000 No effect 0.01
2 2/10,000 Inconvenience Inconvenience 0.2
3 Relatively few failures
5/10,000 0.5
4 1/1,000 Minor dosimetric error
Suboptimal plan or treatment
1.0
5 <0.2% Limited toxicity or under-dose
Wrong dose, dose
distribution, location or
volume
2.0
6 Occasional failures
<0.5% 5.0
7 <1% Potentially serious toxicity or under-dose
10
8 Repeated failures
<2% 15
9 <5% Possible very serious toxicity
Very wrong dose, dose
distribution, location or
volume
20
10 Failures inevitable
>5% Catastrophic >20
25
Major proce
ss
Step Potential
failure mode
Potential causes
of failure
Potential effects
of failure
O S D RPN
Transfer images &
DICOM Data
Transfer Primary data set
Incorrect dataset
associated with
patient
Pull up wrong
patient’s record
Very wrong dose disn
Very wrong vol
1 9 9 81
Phy B 5 9 3 135
Phy C 6 8 7 336
Phy D 3 9 6 162
Phy E 3 9 8 216
Phy F 3 8 3 72
Phy G 5 9 3 135
Phy H 5 9 8 360
Avg 3.88 8.75 5.88
188
FMEA
26
ICARO 20092009-04-28
Page 27Example of a FMEA analysis – Setting up a patient and deliver a treatment at an incorrect position
Potential failure mode
Potential cause(s) of failure Potential effect(s) of failure
O S D RPN Proposed action(s)
Incorrect isocentre
Shift between reference point and isocentre not present in R/V system
Dose delivered to wrong volume, PTV under-dosed and/or OAR overdosed during all fractions
4[1] 10[2] 3[3] 120 Second check of all parameters, review methods
Shift specified incorrectly in set-up instructions
See above 6[4] 10 5[5] 300 Training and second check of all parameters, review methods
Shift specified correctly but made incorrectly
Dose delivered to wrong volume, PTV under-dosed and/or OAR overdosed during one fraction
4 6[6] 3 72 Training, verify table position
Staff omitted to make shift See above 2 6 10 120 Training, verify table position
[1] The value was chosen based on that this do not happens to often. Suggestion is that the data is missing once per 2000 cases. In Lund we have about 2600 patients annually and it seems to be an accurate number that this data is missing not more than once a year.[2] If this will be undetected though out the full treatment the ranking of 10 maybe be adequate.[3] This number is incredible hard to set since it depends very much on the clinical environment. If for example it is possible to acquire the patient position (treatment position) by a simple action by the therapist this can be added to the setup and the patient will be treated “correctly” at each fraction. On the other hand since the data is missing one may step back in the process to get the correct data. Based on the latter process a low number is assigned.[4] This probably occurs much more often than the above case.[5] The delectability is definitely lower in this case compared to the first failure situation.[6] Having a lower severity for this failure mode seems appropriate since there are chances to detect this at the following treatment sessions.
Dublin2010-05-27
Page 28
Quality management tools
Process mapping• Processes and their sub processes
Identify the processes which are more error prone• FMEA (Failure Mode and Error Analysis)• RCA (Root Cause Analysis)
Identify measures for control• SPC (Statistical Process Control)
Measure the quality of the measures/control points• Score actual and potential deviations (e.g.
ROSIS)
Dublin
29
2010-05-27
From Todd Pawlicki
Dublin
30
2010-05-27
From Todd Pawlicki
2010-05-27 Dublin
Page 31Statistical Process Control -
SPC An effective method of monitoring a process
through the use of control charts Control charts distinguish background
variation from events of significance based on statistical techniques • Random variation in the data is de-emphasized
by the way the process behaviour limits are constructed
Using process behaviour charts is an interactive procedure that requires process knowledge and interpretation by the user
Dublin
32
Example of SPC for IMRT verification
2010-05-27
Clin tolerance
Clin tolerance
UCL
LCL𝑋
Todd Pawlicki et al.
SD
SD
Dublin
33
Example of SPC for independent monitor unit check
2010-05-27
Private communication F Nordström
Dublin
34
Four states can be identified
State I• in control and within specifications (IN-IN)• Action required - maintain process control by continuing to monitor the
process State II
• in control and out of specifications (IN-OUT)• Action required - change the equipment and/or procedures of the
process State III
• out of control and within specifications (OUT-IN)• Action required - identify and remove systematic errors in the process
State IV• out of control and out of specifications (OUT-OUT)• Action required - identify/remove systematic errors and/or change the
equipment and/or procedures of the process
2010-05-27
Dublin
35
2010-05-27
From Todd Pawlicki
Dublin
36
QC of IMRT Γ - analysis
2010-05-27 From Létourneau et al 2010 in Med Phys
3% of dose difference (%ΔD) or 2 mm of distance to
agreement (DTA) and an inclusion threshold (%Th) of 10%
Initial model
Initial model
Optimized model
Optimized model
Prostate
Paraspinal SRT
SunN
ucl
ear
MapC
heck
Simple rules to analyse control charts
2011-May ESTRO Anniversary
Page 37
Dublin2010-05-27
Page 38
Quality management tools
Process mapping• Processes and their sub processes
Identify the processes which are more error prone• FMEA (Failure Mode and Error Analysis)• RCA (Root Cause Analysis)
Identify measures for control• SPC (Statistical Process Control)
Measure the quality of the measures/control points• Score actual and potential deviations (e.g.
ROSIS)
Check sheets
What• A pre-defined format for data
collection• Categories are created in advance
but my be added• A mark is added for each example
When To keep track of events/parameters
in a process• Number of correct/incorrect outcome
of certain processes• Number of errors in certain important
parameters
2011-May ESTRO Anniversary
39
Probably suitable for registration the
efficiency in various defence layers/ in your
QA system
We have performed 140 000 MU checks with an
error frequency of 0.003%
ICARO 20092009-04-28
Page 40
What we need?
What errorsorrurs? Or• What incidents?
The frequency One have to
remember – low probability
Learn from each other!!!
A radiotherapy example
2011-May ESTRO Anniversary
41
Thanks Petra Reijnders-Thijssen for the data
Start here Take these later
O – OrganisationH – Human
T – Technical P – Patient
ICARO 20092009-04-28
Page 42
ROSIS - SAFRON
www.rosis.info
Dublin
43
Summary
Using a process characterization scheme, the control chart approach provides better discrimination of process performance than the standard deviation approach.
Control charts also have implications for clinical trials QA where they may be used to appropriately categorize process performance, minimize protocol variations and guide QA process improvements.
A LEAN approach
2010-05-27
Dublin
44
The Problem!!! RT QA guidance is focused on equipment
performance even though most RT events have resulted from human performance failures rather than equipment failure
Lack of adequate guidance for resource allocation• The premise: If it can be checked, check it• That is…..everything
Lack of adequate personnel
2010-05-27
Dublin
45
The Problem (cont)
Pressure on medical physicists to implement new technologies
Lack of guidance on how to implement these technologies
Delays in publishing QA protocols or guidelines for emerging technologies
2010-05-27
Where are the resources and time to do it all?
What should a time starved health worker do?
What to Do?
46Saiful Huq - New Paradigms for QM in RT AAPM Summer School 2011