in-tolerance non-conformance investigations webinar slides
TRANSCRIPT
In ToleranceNon-Conformance Investigations
Presented by
Better by Your Every Measure • 800.828.1470 • www.transcat.com
NCSLI LA/OC Regional Meeting May 12, 2015
To understand:
1. The familiar concept of OOT Measurement Risk
2. The less-familiar concept of In Tolerance Measurement Risk
3. The purpose of Guard Band
4. Compare to your organization’s current definitions• Create definitions for your organization if one doesn’t exist
• Enhance definitions for your organization if one already exists
5. How to apply these concepts to control of your measurement processes in order to reduce/eliminate product false accept/reject situations
Learning Objectives
Measurement Quality Assurance
• Metrological Traceability is important, but . . .
• It means nothing if the industrial processes don’t maintain a good measurement quality assurance program
• MQA is critical to industrial manufacturing processes:– To keep product costs at a minimum
– To maintain safety for employees and product consumers
– To keep product lead times from being delayed
– To make the traceability chain a worthwhile effort (and its costs)
• Am important part of using calibration data to control product decisions is to understand that even an In-Tolerance calibration result can cause False Accept situations on product
Where does Risk Creep into Your Process?
Measurement Quality Assurance encompasses a number of assumptions within a manufacturing process:
1. Product Acceptance You made the right decision about the acceptance or rejection of your product (or a component of your product) during the manufacturing and quality acceptance process.
2. Basis of Decision The decision was based (partially or completely) on quantitative information provided by one or more measuring instruments.
3. Suitability for Intended Purpose
You made the right choice in selecting instrument(s) that are appropriate for the measurements in your process.
4. Instrument Application The instrument(s) were used correctly in the manufacturing process when making decisions about product quality.
Where does Risk Creep into Your Process?
Measurement Assurance encompasses a number of assumptions (con’t):
5. Beginning of Period Reliability
The calibration of the instrument prior to your decision about the product indicated it met its performance expectations.
6. End of Period Reliability
The subsequent calibration of the instrument following your decision about the product indicates that it continues to meet its performance expectations.
7. Calibration Process Calibrations are executed correctly and support the application(s).
8. Non-Conformance Review (NCR)
If a calibration indicates that the instrument failed to meet its performance expectations, your quality system reviewed any impact to the decision(s) made about your product(s) that were based on the quantitative values presented by the instrument. Caution: This may include In Tolerance results in some circumstances
9. NCR Accuracy This impact study was thorough, followed an unbiased, forensic approach, and did not miss the mark on determining any detrimental affect on product.
10. Risk mitigation Any negative result from the impact study was properly mitigated to remove or reduce product risk.
• You’ve got to be kidding, right?
• Sorry but, no –
• But . . . This revelation of another ‘quality blind spot’ is important in protecting product decisions from resulting in a false accept or false reject situation
• Evaluation requires semi-complex concepts relating to sequences of events over time in order to see how this quality blind spot occurs
• An understanding of this particular quality blind spot also requires knowledge of measurement ratios (TAR, TUR, PAR, PUR, etc.)
• Let’s examine this using an example in Medical Device Manufacturing
• We’ll look at product measurements over a one year period
• As we review the measurements, we’ll logically step through the reasoning behind the commonly applied 4:1 ratio
In ToleranceNon-Conformance Investigations
• Heart-Lung Bypass Machine (perfusion equipment)
– Used to handle blood flow through the patient’s body,
bypassing the heart and lungs
– Patient is maintained in a hypothermic state between 28 ⁰C
and 32 ⁰C during the surgery
In ToleranceNon-Conformance Investigations
• Heart-Lung Bypass Machine (perfusion equipment)
– Measuring the product influent and effluent temperatures
– Process tolerance for the effluent is 30 °C ± 2.0 °C
– 30 products measured over a 12 month period
In ToleranceNon-Conformance Investigations
• Heart-Lung Bypass Machine (perfusion equipment)
– “As Produced” measurements taken on dates shown
In ToleranceNon-Conformance Investigations
• Example: Temperature measurement
– Process measurement is 30° ± 2.0° C
– Determine Inspection, Measuring, and Test Equipment (IMTE)
that is suitable for this measurement
LL:
Process
28°C
UL:
Process
32°C30°C
In ToleranceNon-Conformance Investigations
• Process measurement
– An instrument with an accuracy equal to the process limits?
– This is a 1:1 ratio
– Not good because the instrument can drift the full amount over its
cal interval, directly impacting previous process measurements
In ToleranceNon-Conformance Investigations
LL: IMTE
28°C
UL: IMTE
32°C30°C
• Process measurement: 1:1 Ratio
– IMTE was calibrated on 5/12/2014 and adjusted to nominal
– IMTE is used to measure effluent temperature the same day
– Effluent temperature is right at nominal: 30° C
28°C 32°C30°C
LL:
Process
UL:
Process
In ToleranceNon-Conformance Investigations
• Process measurement: 1:1 Ratio
– Instrument is on its last day of use before it will be recalibrated
– Instrument is used to measure effluent temperature 5/11/2015
– Effluent temperature is right at nominal: 30° C
28°C 32°C30°C
LL:
Process
UL:
Process
In ToleranceNon-Conformance Investigations
• IMTE Calibration: 5/12/2015
– IMTE is recalibrated
– As Found reading shows the IMTE drifted to the upper limit: 32°C
– This drift occurred over its calibration interval; instrument adjusted
28°C 32°C30°C
LL: IMTE UL: IMTE
In ToleranceNon-Conformance Investigations
• IMTE Calibration
– Did it move all at once or a little over time?
– How does this impact the measurements of the process since
the last time this IMTE was calibrated?
– Error = IMTE – Standard = 32 – 30 = +2 °C; Correction = -2 °C
28°C 32°C30°C
LL: IMTE UL: IMTE
In ToleranceNon-Conformance Investigations
• OOT-NCR Evaluation
– It must be assumed this shift occurred immediately following the
last calibration because that is the only supporting evidence to
show the last known good condition of its temperature values
– Returning to the product measurement made on 5/12/2014:
28°C 32°C30°C
LL:
Process
UL:
Process
x
In ToleranceNon-Conformance Investigations
• OOT-NCR Evaluation
– The process measurement taken on the day the instrument was
previously calibrated could be incorrect if the instrument shifted in
transportation or handling after the cal was performed
– Applying the correction of 30 °C - 2 °C = 28 °C
– The same applies to the process readings at the end of the cal cycle
28°C 32°C30°C
LL:
Process
UL:
Process
x
In ToleranceNon-Conformance Investigations
• In Tol-NCR Evaluation
– If one process reading had shown the product was at its lower limit:
28 °C, this would have been acceptable for the product test, but . . .
– The In Tolerance cal would not have flagged this situation as ‘at risk’!!
28°C 32°C30°C
LL:
Process
UL:
Process
x
In ToleranceNon-Conformance Investigations
• Here’s a look at the impact to the “As Produced” readings:
In ToleranceNon-Conformance Investigations
Falsely Rejected
Correctly Rejected
Falsely Accepted
• IMTE with better accuracy: 2:1 Ratio
– What if an instrument was selected with an accuracy twice as good?
– If it drifted to its limit but was still In Tolerance, process readings
would only be off by half of the process tolerance
28°C 32°C30°C
LL:
Process
UL:
Process
29°C
LL:
IMTE
31°C
UL:
IMTE
x
In ToleranceNon-Conformance Investigations
• Here’s a look at the impact to the “As Produced” readings:
In ToleranceNon-Conformance Investigations
Falsely Rejected
Correctly Rejected
Falsely Accepted
• Instrument with better accuracy: 4:1 Ratio
– What if an instrument was selected with an accuracy 4X as good?
– If it drifted to its limit, process readings would only be off one quarter
of the process tolerance
28°C 32°C30°C
LL:
Process
UL:
Process
29.5°C
LL:
IMTE
30.5°C
UL:
IMTE
x
In ToleranceNon-Conformance Investigations
• Here’s a look at the impact to the “As Produced” readings:
In ToleranceNon-Conformance Investigations
Falsely Rejected
Correctly Rejected
Falsely Accepted
• IMTE with better accuracy:
– How far can we take this? 10X better? 100X better?
– There are limits of technology
– Eventually it becomes cost prohibitive to go too far with this concept
– A 4:1 ratio generally produces about a 2% consumer risk
– This is widely accepted and practiced
– Depending on the criticality of the product, this may still be too much
In ToleranceNon-Conformance Investigations
• Lesson Learned:– Even “In Tolerance” results can impact your process
measurements!– All cal data must be reviewed against process measurements to
understand the impact to the product (IT-NCR evaluation)– Impact studies are expensive; consumes valuable resources– Can cost thousands of dollars per evaluation event– Parts or product likely have already been released/shipped– Could require product recall– If not released, could require rework– This risk can be reduced– Guard Band the process limits!
In ToleranceNon-Conformance Investigations
• What is Guard Band?
– Determine realistic tolerance limits for the process measurement
– Reduce these limits by the tolerance of the instrument used in the
process to arrive at Lower/Upper Acceptance Limits
30°C28°C
LL:
Process
32°C
UL:
Process
29.5°C 30.5°C
LL:
IMTE
UL:
IMTE
x27.5°C
28.5°C
LAL:
Process
31.5°C
UAL:
Process
x
In ToleranceNon-Conformance Investigations
• Guard Band result:
– Now if the instrument cal reveals a drift up to the maximum tolerance
there is no need to perform IT-NCR evaluation
– If instrument error is greater than this (i.e. it is OOT), then OOT-NCR
28°C 32°C30°C
LL:
Process
UL:
Process
29.5°C 30.5°C
LL:
UUT
UL:
UUT
x
28.5°C
LAL:
Process
31.5°C
UAL:
Process
In ToleranceNon-Conformance Investigations
• Here’s a look at the impact to the “As Produced” readings:
In ToleranceNon-Conformance Investigations
Product will be
corrected BEFORE
it is shipped
Product will be
corrected BEFORE
it is shipped
• Summary:
– Implement a good Measurement Assurance Program
– Understand and exercise good Suitability selection
– Guard Band the process tolerance limits to reduce costly
Non-Conformance investigations
– Understand both the process and the calibration to ensure
the intent of preserving good measurements on the product is
not lost
– Be thorough in your NCR evaluations
– Get help if you’re in over your head!
In ToleranceNon-Conformance Investigations
• In the end, running your business is not only about international, federal, customer, or internal requirements, policies, or procedures – it’s about making a safe, reliable, superior product that fills a need/desire in the market place and is profitable for your company.
• Measurement Quality Assurance should be designed to help you minimize risks in your decision making process about your product’s safety and quality.
• Guard Banding processes is a key part of guaranteeing good measurements!
• Too often MQA is not fully implemented (all 10 categories), causing the reliability of your Measurement Quality Assurance Program to lose value and become ineffective.
• If you’re going to put the effort and money into only some parts of this program, or you simply do not recognize all of the factors, it will likely cost you elsewhere through rework, recall, or consumer perceptions.
• Make your Measurement Quality Assurance Program robust so that it works for you to keep cost and safety issues to a minimum and profits up!
Measurement Quality Assurance/Risk