The importance of Proficiency Testing (PT) & why take part
T&M 2011 Workshops 19 – 20 September 2011
Steve Sidney
Why measure ?
"When you can measure what you are speaking about and express it in numbers you know something about it; but when you cannot express it in numbers your knowledge is of a meagre and unsatisfactory kind: it may be the beginning of knowledge but you have scarcely, in your thoughts, advanced to the stage of science, whatever the matter may be."
British scientist Lord Kelvin (1824-1907)
Quality Measurements
QC / QA
Assuring quality of test & calibration results
• Lab shall have QC procedures
Use of CRM
Participation in PT
Replicate tests
Retesting or recalibration
Correlation of results for different characteristics
• What is Proficiency Testing (PT)
• Goal of PT
• Who provides PT
• Success of PT Programs
• Types of PT
• Educational role of PT
Proficiency Testing (1)
Proficiency Testing (2)
• Features of well designed PTS
• Evaluation of Data
• Why PT is important for labs
• The NLA & PT
Definitions
proficiency testing – evaluation of participant performance against pre-established criteria by means of interlaboratory comparisons
participant – laboratory, organization or individual that receives proficiency test items and submits results for review by the PT provider
proficiency testing provider – organization which takes responsibility for all tasks in the development and operation of a proficiency testing scheme
Goal of PT
• Mechanism to evaluate laboratory performance
• Sample evaluated by labs who participate
• Results submitted and statistically evaluated
• Reports generated and sent to participants
Who provides PT
• Independent 3rd Party
• Regulatory body
• Laboratories who conduct an ILC amongst themselves
Success of PT
• Reduction of non-acceptable results
• Evidence
– CAEL ( Canadian Association of Environmental Analytical Laboratories)
(1995 – 2005 )
– Wisconsin State Laboratory of Hygiene
(1996)
CAEL ( Canadian Association of Environmental Analytical Laboratories) 1995 – 2005
Wisconsin State Laboratory of Hygiene (1996)
Types of PT (1)
• Quantitative
– Numeric
– Statistical result
• Qualitative
– Identify presence (YES/NO)
– Statistics usually not appropriate
Types of PT (2)
• Split sample
– Small groups of labs
• Split level
• Partial process
– Not necessarily only a measurement
Educational Role of PT
• Disadvantage of Pass/Fail approach
• PT should deliver what industry requires
• Should be part of QA & Continuous Improvement
• Educational value has been a by-product
Features of well designed PT (1)
• Covers methodology , testing details & approach
• More than one sample • Series of rounds to cover all aspects of
lab’s capabilities • Excess samples can be used as Ref
Materials • Allow more than one analyst to
participate
Features of well designed PT (2)
• Reports to include sufficient data analysis
• Reports provided timeously
• Sufficient frequency
• Provider to have sufficient resources to provide assistance
• Provide guidelines on estimating measurement uncertainty
Features of well designed PT (3)
• PT Program should cover as much of process as possible
• Conduct follow-up workshops
Evaluation of Results
• Lab submits results
• Data is evaluated against
– All other data submitted
– To ease the burden a score is assigned
– Lab takes action dependent on score
How scoring is performed (1)
• Where the ref value and uncertainties are available make use of
𝐸𝑛 = 𝑥 −𝑋
𝑈 where 𝑈 = expanded uncertainty of
𝑥 − 𝑋
If x and X are uncorrelated then 𝑈 can be replaced
with 𝑘 𝑢2 + 𝑈2 (u= lab and U = ref)
How scoring is performed (2)
• The mean is established
• All results are normalised
• Z scores
SD
uxz
X=lab result, u=mean,
SD=standard deviation
Z-score evaluation
• Z – Scores categorised as follows
3z3
3z2
2z2
Pass
Investigate
Fail
Importance of PT (1)
• Wide difference in lab capabilities
• Skill & capacity challenges
• How to achieve & maintain quality of output
• Many initiatives
• Role of PT
Importance of PT (2)
• Will enable lab to monitor it’s own improvement
• Different objectives can be achieved
• Provides a measure of confidence for customers
• Should not be viewed in isolation of other measures
PT & NLA
• Brief History
• Metrology ILC
• PT
• Pesticide Formulations
• Food
• Water Microbiology
References
• ISO 17025 : 2005
• ISO 17043 : 2010
• ILAC Papers
– Educational Aspects of PT Programs (Flemming, Wruck – 2006)
– Discussion of PT Benefits & Progam Strategies (Middlebrook – 2006)
Questions
For further info regarding the NLA’s PT Programmes contact
Hanlie Badenhorst
[email protected] Tel +(27) 012 349 1500