national association of testing authorities metals workshop/ptnata.pdf · national association of...

Be Absolutely Assuredwww.nata.com.au

National Association of Testing Authorities, Australia

THE USE OF (C)REFERENCE MATERIALS IN TESTING/CALIBRATION AND THE ROLE OF PROFICIENCY

TESTING

Mohan SabaratnamProgram Manager

Reference Material Producers, NATA

National Association of Testing Authorities

NATA PROFICIENCY TESTING REQUIREMENTS

Edita GrujicProficiency Testing and Technical Projects Coordinator, NATA



WhatNATA - Assures Technical Standards across government and private sectors.

WhoNATA is passionate about working for the public good, having a strong beliefthat we can make a difference through our core skills in technical assurance

WhyNATA is the Authority that provides assurance of technical standards through aproven network of best-practice industry experts, for client's who demandcertainty and confidence in the delivery of their products and services



NATA offers services in:

Accreditation

Assessment

Training

Advisory Services



NATA and Proficiency Testing

Update:

• Proficiency Testing field/program policies now available from the NATA website as Technical Circulars with the field/program specific information (except where policies are already defined e.g. NPAAC requirements for Medical testing)

• The NATA Policy Circular #32: Proficiency Testing Scheme Providers Implementation of ISO/IEC 17043: 2010 has been publicised recently



Proficiency Testing• On the international level, ILAC has issued an update to the document

P9: ILAC Policy for Participation in Proficiency Testing Activities

• Effective immediately

• Sets requirements for, and includes guidance to accreditation bodies

• ILAC P9– removes the previous concept of ‘major sub-disciplines’– Introduces the need for facilities to have a ‘PT Plan’ in accordance

with NATA’s requirements for PT coverage



Proficiency Testing

• Revised requirements will be reflected in the updated NATA PT Policy

• NATA will review at assessment facilities’ PT plans

• Other requirements e.g. such as expectations regarding action taken in response to poor performance in PT, any requirements set by regulators or similar bodies etc. will continue unchanged



QUESTIONS FOR DISCUSSION

Open for discussion

THANK YOU!



OVERVIEW OF PRESENTATION• Importance of good measurements• Use of CRM/RM in testing/calibration and proficiency testing (PT)• Common Ground• Practical approaches to CHARACTERISATION• CORE Requirements for Accreditation of RMP• ISO Guide 34 - Management & Technical Coverage• Establishing traceability in the laboratory• ISO/IEC 17025 clauses related to PT & use of CRM/RM• Possible combinations of Reference Material Producers• Unavailability of (C)RM & Selection of a (C)RM• NATA RMP Field Updates• Summary and Conclusions• Useful websites



IMPORTANCE OF GOOD MEASUREMENTSMeasurement results must be fit for their intended use in order toachieve:

• Sustainable development and economic growth;• Environmental integrity of the planet;• Societal equity.

Confidence in measurement results can result in:

• Minimizing Technical Barriers to Trade (TBT);• Sensible decisions based on, for example, food, health and

environmental analyses;• Cost effective spending on measurement infrastructure by

governments.



IMPORTANCE OF GOOD MEASUREMENTS

All measurements must be underpinned by sound metrologicalpractices – method validation, calibration, QA/QC and Traceability/Measurement Uncertainty

MethodValidation

Calibration

QA/QCTraceability/MeasurementUncertainty

Valid measurement



ROLE OF PROFICIENCY TESTING PROGRAMS & USE OF (C)REFERENCE MATERIAL

Proficiency Testing - primary aimProvide a quality assurance tool for individual laboratories to enable them to compare their performance with similar laboratories, to take any necessary remedial action, and to facilitate improvement.A demonstration of competence by a laboratory during an accreditation assessment in accordance with ISO/IEC 17025.

(Certified) Reference Material - primary aimEstablish proper traceability to primary units of measure (eg. SI units) when testing and/or calibration is performed by using (C)RM in:Equipment/instrument calibration;Validation of measurement procedures;Quality control;Establishing proficiency testing samples, where required.

Note: All the above are essential for metrological underpinning of measurement results



COMMON GROUND FOR ESTABLISHING PROFICIENCY TESTING PROGRAMS & PRODUCTION OF

(C)REFERENCE MATERIAL

COMMONALITIES BETWEEN THE TWO PRODUCTION PROCESSES:

Sample stability;Sample homogeneity;Statistical analysis of measurement data;Establishing and Reporting associated measurement uncertainty;Use of SUBCONTRACTORS to establish assigned property;A requirement for gaining and maintaining third party accreditation (ie. NATA)



PRACTICAL APPROACHES TO CHARACTERISATION & PROPERTY (VALUE) ASSIGNMENT ACCORDING TO ISO GUIDE 34 & 35

ISO Guide 34 & 35 recommends possible approaches for characterising a candidate reference material:

• Measurement by a single (primary) method in a single laboratory;

• Measurement by two or more independent reference method in one laboratory;

• Measurement by a network of laboratories using one or more methods of demonstrable accuracy;

• A method-specific approach giving only method-specific assessed property values, using a network of laboratories.

Note: Reference materials established by the above modes have been known to be used as proficiency testing samples.



ACCREDITATION OF REFERENCE MATERIAL PRODUCERS (RMPs)

The core requirements documents are:• ISO Guide 34 (2009) Requirements for the Competence of Reference

Material Producers. • ISO Guide 31 (2000) Reference materials – Content of certificates and

labels.• ISO/IEC 17025 (2005);• NATA RMP Field Application Document

Other related documents include:-• REMCO – ISO Guides 30, 32, 33, 35 and 80 (in preparation)• APLAC – TC 008-2010 Guidelines on the Approach to the

Assessment of Reference Material Producers and the Resulting Scope of Accreditation

• ILAC - G9 (2005) The Selection and Use of Reference Material• ILAC - G12 (2000) Guidelines for the Requirements for the

Competence of Reference Material Producers



ISO/IEC 17025

ISO Guide 34

includes application ofof

ISO Guide 30, 31 & 35

ISO 9000

ISO Guide 34

MANAGEMENT & TECHNICAL COVERAGEOF (CERTIFIED) REFERENCE MATERIAL PRODUCTION



ESTABLISHING (FIT FOR PURPOSE) TRACEABILITY IN THE LABORATORY

Laboratories should be aware that in most cases, all stages of the testing/calibration process shall need validation, and traceability to be established.

Reference materials need to be used on an ongoing basis to ensure integrity in the testing and calibration is maintained.

Laboratories are required to investigate the availability and suitability of reference materials and calibrated artefacts for theiraccredited procedures.

Laboratories are required to determine if the calibrated artefacts orreference materials used in their methodologies provide a level oftraceability that is fit for purpose.



ESTABLISHING (FIT FOR PURPOSE) TRACEABILITY IN THE LABORATORY

Thus, competence of RMP organisations conducting the production or calibration of reference materials/artefacts is a critical step towards obtaining suitable well designed reference materials or artefacts.

Note 1: Accreditation under NATA requires all accredited testing and calibration laboratories tocomply with Clause 5.6, ISO/IEC 17025:2005. It is acceptable that not alwaysmetrological realisation is achievable through established lines of traceability.

Note 2: The concept of traceability of measurement in fields such as chemical and biological sciences is still under international debate so no clear pathway has been accepted.



ISO/IEC 17025 (2005) CLAUSES RELATING TO (C)REFERENCE MATERIALS

• ISO/IEC 17025, clause 4.5 requires laboratories to evaluate suppliers; this includes producers of reference materials and suppliers of calibration services/items.

• Materials purchased by the laboratory is determined by the quality of the producers of these items and the distributor is only part of the supply chain and in most cases only has custody of the material until it is delivered to the user. This is reflected in ISO/IEC 17025 under clause 4.6 Purchasing services and supplies

4.6.2 The laboratory shall ensure that purchased suppies and reagents and consumable materials that affect the quality of test and/or calibrations are not used until they have been inspected or otherwise verified as compying withstandard specifications or requirements defined in the methods for the testsand/or calibrations concerned. These services and supplies used shall complywith specified requirements. Records of action taken to check compliance shallbe maintained.

4.6.4 The laboratory shall evaluate suppliers of critical consumables, supplies and serviceswhich affect the quality of testing and calibration, and shall maintain records of theseevaluations and list those approved.



ISO/IEC 17025 (2005) CLAUSES RELATING TO PROFICIENCY TESTING

5.9 Assuring the quality of test and calibration results5.9.1 The laboratory shall have quality control procedures for monitoring the

validity of tests and calibration undertaken. The resulting data shall berecorded in such a way that trends are detectable and, where practicable,statistical techniques shall be applied to the reviewing of the results. Themonitoring shall be planned and reviewed and may include, but not limited, the following:

(a) regular use of certified reference materials and/or internal quality control using secondary reference materials;

(b) participation in interlaboratory comparision or proficiency-testing programmes;(c) replicate tests or calibrations using the same or different methods;(d) retesting or recalibration of retained items;(e) correlation of results for different characteristics of an item;

NOTE: The selected methods should be appropriate for the type and volume of thework undertaken.

5.9.2 Quality control data shall be analysed and, where they are found to beoutside pre-defined criteria, planned action shall be taken to correct theproblem and to prevent incorrect results from being reported.



POSSIBLE COMBINATIONS OF REFERENCE MATERIAL PRODUCERS (RMPs) Note: The Reference Material Producer (ie The body which is subject to formal accreditation) is the organisation that takes responsibility and authorises the values on the (C)RM certificate - not the actual material/product itself.

ECECECECISO Guide 34Supply & distribution

ISO Guide 34 +ISO/IEC 17025

* Authorisation and Assignment of values on (C)RM and issue of certificates

ECECECISO Guide 34+ISO/IEC 17025

Characterization of Property values

ECECECISO Guide 34Homogeneity & Stability assessment

ECECECECISO Guide 34Material preparation

etcOrg 5Org 4Org 3Org 2Org 1

= Tasks actually performed by each organisationOrg 1-5 etc = Examples only

ISO Requirements Document(s)

Stages and tasks of (Certified) Reference Material [(C)RM)] Production

•The RMP accreditation process assesses the competence for each task.•The RMP itself may perform one or several tasks, as shown above, but must have a quality system that covers all of the tasks involved and the necessary expertise to competently evaluate all the inputs.



WHERE NO REFERENCE MATERIALS EXIST?

ThenThe laboratory shall demonstrate that the alternatives used have:• Sufficient traceability;• Stability;• Homogeneity,• and Accuracy, such that the method and subsequent results can be deemed fit for purpose.

Mandatory use of a specified reference material may be a requirement for some sector schemes or in support of regulatory activities.

There is overlap between ISO/IEC 17025 CALIBRATION accreditation and ISO Guide 34 Reference Material Producer accreditation, in that organisations accredited to these standards can both provide materials that can be used as a reference and to provide traceability for analytical methods.

However, it has been accepted that ISO/IEC 17025 is primarily used for characterisation of candidate reference material whilst ISO Guide 34 encompasses both the characterisation and associated production requirements for the production of a reference material and/or certified reference material.



CHOOSING A REFERENCE MATERIAL?Testing/Calibration laboratories should consider the following points when selecting the service/product they require:

Where the metrological properties of an artefact or instrument have a significant effect on the result of a measurement then calibration is required. This must be performed by a competent laboratory.

Some examples of materials that fall into this category are:Masses used to calibrate balances, Thermometers used for monitoring the temperature of ovens, incubators, Glass beads used for checking sieve pores, Alloy standards for conducting arc-sparktests

Where the laboratory requires a homogenous and stable artefact or if the artefact has the potential for heterogeneity and/or instability, then the testing or calibration laboratory should purchase an artefact from a reference material producer that can demonstrate that these requirements have been met. These material characteristics can also be ascertained by a competent laboratory.

Some examples of materials that fall into this category are:Certified reference cement material, Fatty acid reference material used for GC analysis, A solutionof anions or cations used to calibrate an ICP-MS, Soil with certified dioxin content used for the

validation of a dioxin method, Reference microorganisms cultures for identify and purity characterisation



NATA RMP FIELD UPDATES

• Accreditation granted to SEVENTEEN producers in the areas of organic reference materials used in sports drugs, forensic, therapeutic, reference gases, ethanol standards, grain (wheat & barley) and biological reference cultures;

• Currently, NATA is the largest RMP accreditation body in the world;

• APLAC RMP Technical Document (TC 008) completed. NATA had considerable input;

• APLAC RMP MRA signed in December 2007; Therefore all accredited NATA RMP facilities can use the ‘World Recognised ACCREDITATION’ logo

• Conversion of accredited RMP facilities from ISO Guide 34 (2000)to ISO Guide 34 (2009) is underway.



SUMMARY (1)

1. It is essential that (accredited) labs have access to good quality (C)RMs (which are fit for their interned use).

2. There is process OVERLAPP with the production of reference materials and in developing proficiency testing samples however the primary aim of both process is to be a performance tool in one aspect and establish metrological traceability in another.

3. APLAC has established a MRA for this service (5 signatories – NATA, Australia; CNAS, China; IAJapan, Japan; ACLASS, USA; A2LA, USA.

4. Given the restricted number of RMPs in Australia, reference materials purchased from: RMPs accredited by NATA; RMPs accredited by signatories of the APLAC RMP MRA; National Measurement Institute Australia; other NMIs who are part of the CIPM MRA

are preferred, where available.



SUMMARY (2)

5. The accreditation of RMPs and PT Providers is gaining further momentum and it is hoped that both activities will be included underthe ILAC Arrangement in the near future. The laboratory community and accreditation bodies are working closely in finalising this work.

6. These activities will provide further confidence in laboratory outputs (i.e. measurement results) and thus provide socio-economic benefits tothe community at large.



USEFUL WEB-SITES (REFERENCE MATERIALS)

Whilst no single source of information on the availability of reference materials currently exists, laboratories are recommended to contact reference material producers and distributors to discuss availability and suitability.

In addition, laboratories may find it useful to refer to the COMAR database for availability of reference materials. COMAR is a non-commercial network of national and international organisations, which is funded by BAM. The website address is www.comar.bam.de.

Other sources of information are:NATA www.nata.com.au REMCO: www.iso.org/remcoAPLAC www.aplac.org Springer (ACQUAL): http://link.springer.deILAC www.ilac.org Euromet: www.euromet.orgBIPM www.bipm.fr Eurolab: www.eurolab.orgCITAC www.citac.cc Eurachem: www.eurachem.orgEA www.european-accreditation.org

Other National Accreditation Body websites



THANK YOU FOR YOUR KIND ATTENTION

“The scientist of today think deeply instead of clearly. One must be sane to think clearly, but one can think deeply and be quite insane.”

-Nicola Tesla-

Wikipedia

The International System of Units (SI) measuring magnetic field B (also referred to as the magnetic flux density and magnetic induction) is the TESLA, was named in his honour at the Conférence Générale des Poids et Mesures, Paris, 1960.



QUESTIONS FOR DISCUSSION

1) What has been your experiences in getting appropriate CRMs?

2) Have you tried using a PT sample as a reference material?

3) How do you deal with CRMs that are in a different matrix to that of the sample?

4) In general, your experiences with NATA assessments and the composition of assessment teams?

5) How do you select your CRM producer?

national association of testing authorities metals workshop/ptnata.pdf · national association of...

Documents