reporting results and assessment of compliance€¦ · reporting results and assessment of...

Reporting results and

assessment of compliance

Eurachem workshop Quality in Analytical Measurements

Lisbon, 19‐21 May 2014

Bertil Magnusson

The measurement cycle

Client issue

Client

Define issue

Decision on

measurement

Client

Sampling

Analysis

Analytical chemist

InterfaceReport on

measurement

Decision on result

Evaluation

Data evaluation

Analytical chemist

2014-05-19Reporting results and compliance 2

Content

Reporting results•What have been measured

• The measurand including the sample, the test item

•The sampling target – laboratory sample or ?

•ISO recommendations for reporting

•Reporting and number of significant figures

•Info about quality/accreditation and method/procedure

For compliance assessment we need•External or internal specification/legal limit etc

•Measurement result

•Decision rule

Summary

2014-05-19 3Reporting results and compliance

Correct reporting of results

accurately, clearly, unambiguously and objectively,

and in accordance with any specific instructions in the test …method.

(ISO/IEC 17025)

Corrected reported we may compare results and assess compliance


Specification of measurand (VIM3*)

Measurand ‐ quantity intended to be measuredNOTE 1 The specification of a measurand requires knowledge of the•kind of quantity•description of the state of the phenomenon, body, or substance carrying the quantity including any relevant component, •the chemical entities involved.

Example of a measurand specification

mass fraction of total Cd in mg/kg in acertain soil batch

reported on dry mass basis (105 °C, 2h)


Measurement object/test item

*ISO Guide 99 (2007) International Vocabulary of Metrology – Basic and General Concepts and Associated Terms VIM 3rd edition, ISO, Geneva

Sampling target


Some ISO recommendations for reporting measurement results

For writing ISO standards the following is recommended regarding reporting results*

*ISO/IEC Directives, Part 2, Edition 6.0, Rules for the structure and drafing of Inernational Standards, ISO/IEC, Geneva, (2011).

Issue Recommended Not recommended

Clear indication of unit 23 °C ± 2 °C or (23 ± 2) °C

23 ± 2 °C

Separate informationand unit

the water content is20 ml/kg and not “

20 ml H2O/kg or 20 ml of water/kg

Do no mix units (230 ± 11,5) V 230 V ± 5 %


How to report high uncertainties

For analysis with high uncertainty

e.g. result 1 % lower 0,5 % and higher 2 %

I report ca 1 %


Discussion with your colleagues –How to avoid gross errors in reporting

Which questions shall i ask myself?

Think about you own experience!

What errors could we avoid?

I will later give you some examples of errors


How to report

If we now trust

definition of the measurand

the result and

its uncertainty


Reporting results and significant figures

It suffice to quote uc(y) and U to at most two significant digits, although in some cases it may be necessary to retain additional digits to avoid round‐off errors in subsequent calculations ” (GUM 7.2.6)

The number of decimals for the result and the uncertainty is the same. Example from Eurachem Uncertainty guide

Total nitrogen: (3.52 ± 0.14) g/100 g *

*The reported uncertainty is an expanded uncertainty calculated using a coverage factor of 2 which gives a level of confidence of approximately 95 %.


Base for reporting?

Basis for reporting

For some hygroscopic sample types the basis for reporting is crucial

Some examples – Result is reported on:

•dry mass basis (105 °C 2h)

•dry mass basis (80 °C 2h) according to ISO xxx

•sample equilibrated according to ISO xxx

•ignited basis (800 °C 1h)

And if no treatment is performed it might be importatnt to state that

Example of wording is here ”as reveived” or ”as is”


Info about quality/accreditation and method/procedure

For accredited methods the accreditation body logo is mandatory for all test results issued by the laboratory

According to ISO/IEC 17025 – regarding reporting results

reference to the sampling plan and procedures used by the laboratory or other bodies where these are relevant to the validity or application of the results;


Content

Reporting results•What have been measured

• The measurand including the sample, the test item

•The sampling target – laboratory sample or ?

•ISO recommendations for reporting

•Reporting and number of significant figures

•Info about quality/accreditation and method/procedure

For compliance assessment we need•External or internal specification/legal limit etc

•Measurement result

•Decision rule

Summary


Compliance

According to ISO/IEC 17025 a test report shall, where necessary for the interpretation of the test results, include:

…..

•where relevant, a statement of compliance/non‐compliance with requirements and/or specifications

•where applicable, a statement on the estimated uncertainty of measurement; information on uncertainty is needed …when the uncertainty affects compliance to a specification limit;

•….


Eurachem guide on compliance by Uncertainty and Traceability WG


Four results with uncertainty and an upper limit


UpperLimit

(i)Result plus expanded uncertainty above limit

(ii)Result above

limit: limit within

expanded uncertainty

interval

(iii)Result below

limit; limit within


interval

(iv)Result minus

expanded uncertainty below limit

UpperLimit


(ii)Result above

limit: limit within


interval

(iii)Result below

limit; limit within


interval

(iv)Result minus


UpperLimit


(ii)Result above

limit: limit within


interval

(iii)Result below

limit; limit within


interval

(iv)Result minus


Lets us first look at terminology

Terminology

• decision rule: a documented rule that describes how measurement uncertainty will be allocated with regard to accepting or rejecting a product according to its specification and the result of a measurement.

• acceptance zone: the set of values of a characteristic, for a specified measurement process and decision rule, that results in product acceptance when a measurement result is within this zone.

• rejection zone: the set of values of a characteristic, for a specified measurement process and decision rule, that will give non‐compliance when a measurement result is within this zone.

• guard band, g: the magnitude of the offset from the specification limit to the acceptance or rejection zone boundary

• decision limit – the point between the rejection and acceptance zone


Lets look at a legal case

Blood alcohol ‐ a result clearly over an upper limit

Sample taken from a driver in Sweden andanalysed at a laboratory

Analytical Report •Mass fraction of ethanol in blood sample is

0,221 mg/g ± 0,013 (k=2)

Limit in Sweden is 0,200 mg/g(20 mg/100g)


Will this driver be punished?

Using a decision rule we calculate an acceptance zone


Acceptance zone with decision limits

Product/item to be tested(Sampled and analysed)

Result ± U & Permitted Limit or Specification

& Decision rule

In legal cases we must be very sure (> 99 %) that the “true” value of the measurand is OVER the permitted limit.

Regarding a product specification we must be sure (>95 %) that the value of the measurand is BELOW an upper specification.

Acceptance zone for a legal case with an upper limit

Upper limit is 0.200 mg/gFrom decision rule is calculated a guard band.The guard band is 0.02 mg/g

Blood alcoholMeasurement result 0,221 mg/g

*

*


We repeat this example – Blood sample from a driver to be tested for alcohol content • Measurement of blood alcohol where we want to be sure to only punish those drivers that truly has exceeded the limit

• Measurement procedure: Ethanol in blood by Head‐Space GC

• Quality routine: Two independent results – different analysts using different instrument ‐Max difference (range) 0,003 mg/kg

1.Measurand – Massfraction total EtOH in a laboratory blood sample

2.Analytical result ‐ C (EtOH)= 0,221 mg/g

3.Uncertainty – U = 0,013 mg/g, k=2 (95 %), combined standard uncertainty, uc is 0.0065 mg/g.

4.“Specification” – Upper permitted limit 0,200 mg/g

5.Decision rule ‐ The decision limit is the concentration above which it can be decided with a statistical certainty of about 99.9 % (alfa =0,001) that the permitted limit has been truly exceeded


For calculation of guard band - one sided 99.9 % t = 3.1

Probability that the true value of the measurand is over the limit is high (about 99.9 %)

mg/g020,0065.01.3

1.3

cug


Decision Limit

Upper Limit

Ethanol concentrationmg/g

0,200 0,220=0.200 + 0.020

*

≈ 0,1 % ≈ 99,9 %

Blood alcoholMeasurement result 0,221 mg/g*

Calculation of guard band, g-

g

What do we need for compliance assessment?

1. A measurand clearly specified including the measurement object/test item

2. An analytical result

3. The uncertainty – For an expanded uncertainty the k factor and the corresponding confidence level should be stated e.g. k = 2 for 95 % confidence

4. A specification giving upper and/or lower limits

5. A decision rule


Example of a decision rule

Decision rule

The batch will be considered to be non‐compliant if the probability of the value of the concentration being greater than the upper limit exceeds 95%

1.A specification of the measurand including the object/test item Batch

2.An analytical result ‐ Single value, mean value, each single value?Mean value

3.An uncertainty ‐ Normally an expanded uncertainty at 95% confidence level

4.A specification giving upper and/or lower permitted limitsUpper limit

5.A decision rule how to take measurement uncertainty into accountNon‐compliant if probability for out of specification is higher than 95%


Example ‐ Sludge from water purification plants

Sludge from water purification plants can be used for soil improvement. One of the toxic metals that can be a problem is cadmium. The upper limit on the total cadmium in sludge is set to 2 mg/kg.

1.Measurand – Mass fraction of cadmium, Cd, in a consignment delivered to a customer

2.Analytical result ‐mass fraction (Cd) = 1.82 mg/kg

3.Uncertainty – U = 0.20 mg/kg, k = 2 (95 %). Standard uncertainty, u = 0.10 mg/kg. The uncertainty includes both sampling and analytical uncertainty

4. Specification – Upper permitted limit 2.0 mg/kg

5.Decision rule ‐ The decision limit is the mass fraction where it can be decided with a confidence level of approximately 95 % (α=0.05) that the batch has a mass fraction below the upper limit.


Conformity assessment – Cd result 1.82 mg/kg

• The guard band is calculated as 1.65u = 0.165 mg/kg with k value 1.65 for one tailed t value at 95 % confidence. The decision limit will be 2 – 0.165 = 1.84 mg/kg. All values below this value are in the acceptance zone. All values equal to or above are in the rejection zone.The sludge sample meets the compliance requirements.


Decision limit

Upper limit

One sided and two sided confidence intervals

Upper permitted limit

Then decision limit at:

95 % result + 1,6 u

99,9 % result + 3,09 u

where u is the standard uncertainty


0.0

0.1

0.2

0.3

0.4

-3s -2s x +2s +3s

Analytical result

Rel

ativ

e pa

rt of

the

resu

lts

_x ± 3s

(99.7% )x ± 2s

(95.4% )

_

_

Measurement result

Confidence interval

95 % result ± 2 u

Acceptance & rejection zones for a legal case

*

Case: An upper permitted limit (legal case).Decision rule – for non compliance , rejection we should be very sure that the true value of the measurand is greater than the limit


Example of Toy testingaccording to

EN 71-3 Safety regulations –Part 3: Migration of certain elements

This standard refers to Eurachem Guide for calculation of the guard band. In this case the guard band is used

as an analytical correction on the results

The analytical correction of the test result is 30 to 60 % before

comparing the test result with a limit value

Assignment – test of a perambulator

Parts sampled according to EN1888 and test of parts according to European standard EN 71‐3. Toys ‐Safety regulations ‐ Part 3: Migration of certain elements.

• A = White paint coating

• B = White plastic

• C = Blue plastic

After leaching the elements were determined by inductively coupled plasma optical emission spectrometry (ICP‐OES).

Slide 312014-05-19Reporting results and compliance

Results

EN 71‐3: “The results are expressed as mg soluble element/kg material after analytical correction according to EN 71‐3 section 4.2 (Interpretation of results).”

Slide 32

A B CRegulatory&Decision

limitmg kg-1

Antimony, Sb <5 <5 <5 60Arsenic, As <5 <5 <5 25Barium, Ba 273 <5 <5 1000Cadmium, Cd <5 <5 <5 75Chromium, Cr <5 <5 <5 60Lead, Pb <10 <10 <10 90Mercury, Hg <5 <5 <5 60Selenium, Se <25 <25 <25 500

Compliance? All the tested parts fulfil the requirements according to EN 71-3

2014-05-19Reporting results and compliance

Measurement cycle


Client Issue

Client

Define issue

Decision on

measurement

Client

Sampling

Analysis

Measuring scientist

InterfaceReport on

measurement

Decision on result

Evaluation

Data presentation

Measuring scientist

Use of uncertainty information in compliance assessment

Eurachem 2007

Which parameter is relevant for compliance

Compliance with a legal limit or an external specification

uc combined standard measurement uncertainty,

Compliance in production – is my process stable

sRw where sRw is the intermediate precision, the within laboratory reproducibility standard deviation

Comparison in research – is there any difference between sample A and treated sample B

sr where sr is the repeatability standard deviation


SummaryWhat do we need for compliance assessment?

1. A measurand clearly specified including the measurement object/test item

2. An analytical result reported with all information, e.g.

mass fraction of total Cd is 1.82 mg/kg ± 0.20 mg/kg in asoil batch xxx, 2014‐05‐20

reported on dry mass basis (105 °C, 2h)3. The relevant standard “uncertainty”

4. A specification giving upper and/or lower limits

5. A decision rule

6. From this we calculate acceptance and rejection zones


Acceptance and rejection zones


Measurement cycle


Client Issue

Client

Define issue

Decision on

measurement

Client

Sampling

Analysis

Measuring scientist

InterfaceReport on

measurement

Decision on result

Evaluation

Data presentation

Measuring scientist

Use of uncertainty information in compliance assessment

Eurachem 2007

Examples

• Is the result possible?

• Did I use the correct unit?

• Did I analyse the correct sample?

• Did I followed the SPO?

• Did I use the correct dilution factor?

• Have I included the information about the used procedure?

• Did I correct for mass loss?


reporting results and assessment of compliance€¦ · reporting results and assessment of...

Documents