smokeless tobacco sub-group - coresta · smokeless tobacco sub-group ... karl wagner, ph.d. altria...

Smokeless Tobacco Sub-Group

Technical Report

CORESTA Reference Products

2014 Analysis

January 2015

Author and Sub-Group Secretary:

Karl Wagner, Ph.D.

Altria Client Services Inc., U.S.A.

Sub-Group Coordinator and Study Coordinator:

John Bunch, Ph.D.

American Snuff Company, U.S.A.

Co-Author and Statistical Analysis:

Michael Morton, Ph.D.

Altria Client Services Inc., U.S.A.

STS-CTR 2014 Analysis of CRPs – January 2015 2/45

Table of Contents

Table of Contents ....................................................................................................................... 2

1. Summary ............................................................................................................................ 3

2. Introduction ........................................................................................................................ 3

2.1 Objective ...................................................................................................................... 4

3. Organisation ....................................................................................................................... 4

3.1 Participants .................................................................................................................. 4

3.2 Protocol ........................................................................................................................ 5

3.2.1 Sample Shipment .................................................................................................. 5

3.2.2 Within Laboratory Sample Preparation ................................................................ 5

3.2.3 Sample Analysis and Data Reporting ................................................................... 5

4. Data – Raw ......................................................................................................................... 6

5. Data – Statistical Analysis .................................................................................................. 9

5.1 Exclusion of Outliers ................................................................................................... 9

5.2 Calculation of Repeatability (r) and Reproducibility (R) .......................................... 10

5.3 Calculation of Z-Scores ............................................................................................. 11

5.4 CRP Stability Assessment ......................................................................................... 12

6. Data Interpretations .......................................................................................................... 14

7. Recommendations ............................................................................................................ 15

APPENDIX A: Raw Data Plots for 2014 WG4 Study ............................................................ 16

APPENDIX B: CRP Stability Assessment, Mean Data Plots ................................................. 30

APPENDIX C: Results using Cochran’s Test for Outlier Detection ...................................... 44


1. Summary

In May 2014, the CORESTA Smokeless Tobacco Sub-Group (STS) conducted the annual

Working Group 4 (WG4) inter-laboratory study designed to assess the stability of the four

CORESTA Reference Products (CRPs). Eleven laboratories participated in the 2014 study.

The participating laboratories reported the levels of nicotine, pH, moisture (oven volatiles),

and tobacco specific nitrosamines (TSNAs) using CORESTA Recommended Methods

(CRMs). In an initial study conducted in 2010, the STS recommended these analytes as

appropriate markers for monitoring product stability. Tabulated data are presented along with

repeatability (r) and reproducibility (R) and z-scores.

The results from the stability analysis over all study years showed only a few trends.

Specifically:

• In the 2012/2013 report1, a statistically significant increase in moisture was noted for

CRP3 over the study years (2010-2013). This upward trend did not continue in 2014;

however, the 2014 CRP3 moisture results are statistically significantly higher as

compared to 2010 and 2011.

• In the 2012/2013 report, an increase in inter-laboratory variability for pH was noted

for CRP3 over the study years (2010-2013). There is still an indication that this is the

case; however, the trend did not reach the level of statistical significance in 2014.

• In 2012, there was a notable increase in inter-laboratory variability for moisture for

CRP4. This year the lab-to-lab differences were much smaller, possibly because all of

the laboratories ordered the CRPs from North Carolina State University just prior to

testing.

Based on the results taken in their entirety, the conclusion of this study is that all four CRPs

are suitable for continued use as reference products. Lack of any clear and significant trends

in the stability results indicate that storage at −20 °C is an appropriate storage condition for

the four reference products. The recommendations for 2015 are to continue to monitor the

stability of the CRPs and to continue to require all participating laboratories to procure the

CRPs immediately prior to the laboratory phase of the study to mitigate differences in long

term storage conditions where the products may not have been stored at −20 °C.

2. Introduction

In November 2008, the Smokeless Tobacco Sub-Group (STS) was established by

recommendation of the CORESTA Scientific Commission. In 2009, STS Working Group

Three (WG3) cooperated to design and manufacture four CORESTA Reference Products

(CRPs) referred to as CRP1, CRP2, CRP3, and CRP4. These products were intended as

replacements for the Smokeless Tobacco Research Products: 2S3 (Moist Snuff), 1S2 (Dry

Snuff) and 2S1 (Loose-leaf Chewing Tobacco), which were more than ten years old.

At the Amelia Island, Florida, STS meeting (October 2009), Working Group 4 (WG4) was

organized to proceed with the chemical characterization of the four CRPs. The protocol for

the first WG4 study2 was distributed in December 2009 and the study was conducted in 2010.

This study included 43 analytes and did not specify methods of analysis. Many of the results

from this study showed a wide range in analyte levels, which is not unexpected since the

methods were not harmonized. The recommendations from the 2010 study were to monitor

the stability of the CRPs on an annual basis, by determining the levels of nicotine, pH,

1 STS Technical Report: CORESTA Reference Products 2012 and 2013 Analysis, July 2014.

2 STS Technical Report: CORESTA Reference Products 2010 Analysis, February 2014.


moisture (oven volatiles) and TSNAs using CORESTA Recommended Methods (CRMs) or

draft CRMs. This annual analysis would allow the STS to determine when the CRPs should

be remanufactured.

At the STS Meeting in Hamburg, Germany (May 2011), it was decided to adopt the

recommendations from the 2010 WG4 study and initiate the first annual analysis of the four

CRPs using CRMs or draft CRMs. The first WG4 study using CRMs or draft CRMs was

conducted in 2011, and a similar study has been carried annually since that time. The focus of

this report is the 2014 study. Repeatability and reproducibility (r & R) and z-scores were

calculated for the 2014 study while the stability analysis includes data from 2010-2014. As

mentioned earlier, methods of analysis were not specified for the 2010 WG4; however, the

data were considered to be similar enough for inclusion in the analysis.

The CRPs continue to be stored at -20°C and distributed by the North Carolina State

University (NCSU) Tobacco Analytical Services Lab under the direction of Dr. Ramsey

Lewis3 and Karen Andres

4.

2.1 Objective

The 2014 participating laboratories were to provide analytical results for pH, moisture (oven

volatiles), nicotine, and the four TSNAs (N-nitrosonornicotine (NNN), N-nitrosoanatabine

(NAT), N-nitrosoanabasine (NAB) and 4-(N-nitrosomethylamino)-1-(3-pyridyl)-1-butanone

(NNK)). This work was conducted using the applicable CRMs referenced in Section 3.

These studies were conducted to support the assessment of stability of the CRPs and to

provide an assessment of inter-laboratory variability. Data were collected from the

participating laboratories and statistically evaluated in basic conformance with the

recommendations of ISO 5725-2:1994 and ISO/TR 22971:2005. The stability analysis was

conducted using the study data from 2010 through 2014. Additionally, z-scores were

calculated for the 2014 data as an additional measure of each laboratory’s performance as

compared to the results of other laboratories.

3. Organisation

3.1 Participants

A list of the participating laboratories is provided in Table 1. The laboratories are listed in

alphabetical order. Letter codes were assigned to each laboratory and do not correspond to

the order in the table below. Since the statistical analysis covers several years, the laboratory

letter codes were maintained between the tables for all years. Furthermore, not all

laboratories participated in all time points or submitted data for all analyses.

Table 1: List of Participating Laboratories in the 2014 WG4 Study

2014 WG4 Participants

Altria Client Services

American Snuff Company

Essentra Scientific Services

Global Laboratory Services

Imperial Tobacco Group, Seita

3 [email protected]

4 [email protected]


2014 WG4 Participants

Japan Tobacco, Inc.

R.J. Reynolds Tobacco Company

Swedish Match Northern Europe

Swisher International, Inc.

Technical Center of Shanghai Tobacco Group Co., Ltd

University of Kentucky

3.2 Protocol

Specific details from the protocol are described below:

3.2.1 Sample Shipment

Laboratories were responsible for procuring approximately 150g of each of the CRPs from

NCSU immediately before starting the 2014 study. Laboratories were requested to store the

samples at approximately 4°C upon receipt if the analyses would be conducted within one

week or to store the samples at approximately -20°C if the analyses would be delayed. The

study start date was May 4, 2014. Laboratories were requested to submit data by May 23,

2014. The samples are identified in Table 2.

Table 2: Sample Identification

Product Type

CRP1 - Swedish style snus pouch

CRP2 - American-style loose moist snuff

CRP3 - American-style loose dry snuff powder

CRP4 - American-style loose-leaf chewing tobacco

3.2.2 Within Laboratory Sample Preparation

The laboratories were directed to remove the samples from cold storage the day before testing

and to not open the samples until equilibrated to ambient temperature. Once samples were

opened, they could be stored at approximately 4°C for up to one week. Additionally:

• The snus pouch (CRP1) was to be cut into two halves and placed directly into the

extraction vessel. Both the tobacco and pouch material were included in the analysis.

• The moist snuff reference product (CRP2) and the dry snuff reference product (CRP3)

did not require sample grinding and were analyzed as received.

• The Loose Leaf reference product (CRP4) was to be ground according the

participating laboratory’s standard procedure.

3.2.3 Sample Analysis and Data Reporting

The WG4 participating laboratories were instructed to conduct triplicate analyses for the

following: pH, moisture (oven volatiles), nicotine, and TSNAs. The laboratories were

requested to use the following CRMs:

• pH: CRM N° 69, Determination of pH in Smokeless Tobacco Products

• Moisture (oven volatiles): CRM N° 76, Determination of Moisture Content (Oven

Volatiles) of Smokeless Tobacco Products


• Nicotine: CRM N° 62, Determination of Nicotine in Tobacco and Tobacco Products

by Gas Chromatographic Analysis

• TSNAs: CRM N° 72, Determination of Tobacco Specific Nitrosamines in Smokeless

Tobacco Products by Liquid Chromatography - Tandem Mass Spectrometry

Participating laboratories were requested to document any deviations from the CRMs and

submit the deviations with their results; however, no deviations were reported. All test results

were to be reported on an as-is basis with no correction for moisture content. The results

were not to be rounded and ideally reported to at least one more digit than typically required.

The study results and the comments were to be sent by e-mail to John E. Bunch, Study

Coordinator.

4. Data – Raw

The full data set for the 2014 study is listed in Table 3. The results are presented on an as-is

basis, without correction for moisture. Each analysis includes three replicates. Raw data

plots that include all replicates, without removal of outliers, are given in Appendix A.

Table 3: Full Data Set for 2014 WG4 Study (results are presented on an as-is basis)

Lab Code

Product Nicotine pH Moisture NNN NNK NAT NAB

% % µg/g µg/g µg/g µg/g

A CRP1 1.01 7.91 49.83 0.67 0.23 0.53 0.035

A CRP1 0.96 7.94 50.48 0.70 0.24 0.59 0.041

A CRP1 1.04 7.94 49.64 0.61 0.22 0.57 0.040

A CRP2 1.29 7.67 54.85 1.82 0.50 1.90 0.164

A CRP2 1.30 7.66 54.72 1.85 0.46 1.94 0.177

A CRP2 1.28 7.66 54.88 1.76 0.50 1.94 0.178

A CRP3 2.21 6.59 8.96 8.50 4.85 6.67 0.404

A CRP3 2.18 6.67 9.94 8.29 4.76 6.06 0.410

A CRP3 2.19 6.65 8.81 8.77 4.52 6.58 0.383

A CRP4 1.06 5.89 22.62 1.97 0.46 1.43 0.064

A CRP4 1.06 5.90 22.65 2.18 0.47 1.32 0.069

A CRP4 1.09 5.90 21.93 1.98 0.46 1.29 0.075

F CRP1 0.88 7.76 51.75 0.69 0.21 0.54 0.033

F CRP1 0.73 7.78 52.20 0.68 0.20 0.50 0.029

F CRP1 0.77 7.81 50.32 0.70 0.21 0.52 0.034

F CRP2 1.19 7.641

54.58 1.97 0.47 1.78 0.146

F CRP2 1.17 7.511 54.44 1.80 0.45 1.72 0.147

F CRP2 1.17 7.551 54.42 1.81 0.46 1.81 0.134

F CRP3 2.15 6.51 8.88 8.50 5.22 6.07 0.368

F CRP3 2.13 6.56 8.91 8.62 4.52 6.08 0.372

F CRP3 2.17 6.59 8.81 8.62 4.73 5.50 0.377

F CRP4 1.11 5.61 23.04 1.81 0.39 1.16 0.049

F CRP4 0.95 5.75 23.45 1.96 0.54 1.23 0.057

F CRP4 0.81 5.72 21.37 1.89 0.42 1.22 0.057

G CRP1 1.00 7.99 52.33 0.80 0.23 0.64 0.044

G CRP1 0.99 7.92 52.44 0.72 0.21 0.57 0.036

G CRP1 1.03 7.95 52.73 0.73 0.21 0.58 0.039

G CRP2 1.24 7.71 54.67 2.06 0.48 2.04 0.187


Lab Code



G CRP2 1.27 7.70 54.63 2.10 0.47 1.99 0.188

G CRP2 1.26 7.70 54.65 2.10 0.49 2.00 0.191

G CRP3 2.12 6.72 8.91 9.60 3.96 6.53 0.500

G CRP3 2.21 6.72 8.93 9.23 3.98 6.49 0.509

G CRP3 2.14 6.72 8.94 9.49 3.91 6.51 0.483

G CRP4 1.07 5.92 21.23 2.18 0.46 1.40 0.072

G CRP4 1.06 5.92 21.19 2.26 0.41 1.33 0.076

G CRP4 1.07 5.92 21.27 2.26 0.44 1.33 0.067

I CRP1 0.98 7.95 53.67 0.66 0.22 0.54 0.030

I CRP1 0.97 7.95 51.55 0.71 0.22 0.52 0.031

I CRP1 0.92 7.95 51.20 0.70 0.21 0.53 0.032

I CRP2 1.25 7.68 53.70 2.01 0.48 1.79 0.160

I CRP2 1.24 7.67 53.45 1.97 0.46 1.83 0.160

I CRP2 1.25 7.67 53.52 1.94 0.48 1.89 0.157

I CRP3 2.13 6.85 7.67 8.98 4.31 5.53 0.396

I CRP3 2.13 6.84 7.45 8.78 4.23 5.42 0.408

I CRP3 2.14 6.84 7.36 8.47 4.46 5.71 0.389

I CRP4 1.17 5.90 22.03 2.15 0.50 1.33 0.062

I CRP4 1.17 5.89 21.95 2.15 0.50 1.36 0.062

I CRP4 1.15 5.89 21.99 2.17 0.47 1.33 0.058

L CRP1 1.02 7.89 51.41 0.58 0.20 0.48 0.037

L CRP1 1.04 7.92 51.42 0.61 0.21 0.50 0.037

L CRP1 1.03 7.91 49.54 0.63 0.21 0.50 0.036

L CRP2 1.29 7.68 54.59 1.75 0.44 1.69 0.146

L CRP2 1.27 7.69 54.49 1.74 0.44 1.70 0.150

L CRP2 1.28 7.69 54.44 1.73 0.43 1.70 0.150

L CRP3 2.21 6.72 8.44 8.57 4.31 5.82 0.406

L CRP3 2.21 6.72 8.56 8.49 4.24 5.77 0.395

L CRP3 2.20 6.71 8.50 8.77 4.18 5.78 0.403

L CRP4 1.07 5.93 23.28 1.95 0.44 1.20 0.063

L CRP4 1.07 5.93 23.27 1.93 0.43 1.18 0.066

L CRP4 1.07 5.93 23.28 1.93 0.43 1.17 0.061

M CRP1 0.98 8.04 48.61 0.58 0.21 0.49 0.028

M CRP1 0.98 7.95 47.75 0.68 0.23 0.53 0.032

M CRP1 1.01 7.93 48.16 0.66 0.21 0.52 0.029

M CRP2 1.29 7.69 54.88 1.77 0.43 1.72 0.152

M CRP2 1.33 7.70 54.22 1.78 0.47 1.75 0.153

M CRP2 1.32 7.71 54.29 1.79 0.47 1.81 0.145

M CRP3 2.20 6.84 10.28 8.44 4.06 5.79 0.397

M CRP3 2.13 6.83 10.39 8.51 4.12 5.63 0.421

M CRP3 2.15 6.84 10.12 8.10 4.16 5.67 0.398

M CRP4 1.06 5.89 21.44 1.83 0.39 1.27 0.055

M CRP4 1.05 5.91 21.80 1.93 0.46 1.30 0.061

M CRP4 1.13 5.94 22.07 1.75 0.40 1.16 0.059

O CRP1 0.88 7.57 54.34 0.65 0.20 0.52 0.018

O CRP1 0.88 7.57 54.13 0.64 0.21 0.53 0.016


Lab Code



O CRP1 0.86 7.58 54.24 0.62 0.20 0.52 0.015

O CRP2 1.16 7.48 54.47 1.66 0.44 1.72 0.109

O CRP2 1.15 7.48 54.41 1.69 0.45 1.76 0.112

O CRP2 1.12 7.49 54.39 1.69 0.46 1.75 0.112

O CRP3 1.95 6.72 10.31 5.76 3.18 4.72 0.225

O CRP3 1.98 6.75 10.33 6.99 3.81 5.73 0.294

O CRP3 1.94 6.73 10.18 5.90 3.13 4.65 0.234

O CRP4 0.93 5.94 21.61 1.65 0.43 1.28 0.042

O CRP4 0.99 5.97 21.79 1.93 0.54 1.49 0.061

O CRP4 0.99 5.96 21.40 1.94 0.44 1.37 0.041

P CRP1 0.89 7.31 50.93 0.69 0.18 0.44 0.032

P CRP1 1.00 7.33 51.04 0.70 0.18 0.44 0.033

P CRP1 0.87 7.35 51.23 0.69 0.18 0.46 0.035

P CRP2 1.18 7.42 53.14 2.00 0.40 1.49 0.158

P CRP2 1.20 7.44 52.77 2.09 0.40 1.47 0.150

P CRP2 1.26 7.47 53.31 1.98 0.39 1.47 0.153

P CRP3 2.05 6.83 8.02 8.86 5.02 6.07 0.432

P CRP3 2.05 6.85 8.49 8.58 4.99 5.91 0.435

P CRP3 2.04 6.82 8.17 8.72 5.05 5.93 0.432

P CRP4 1.23 5.93 21.13 2.18 0.48 1.39 0.069

P CRP4 1.22 5.94 20.98 2.15 0.47 1.39 0.069

P CRP4 1.20 5.92 21.12 2.19 0.48 1.37 0.070

Q CRP1 0.97 7.96 50.20 0.63 0.21 0.45 0.023

Q CRP1 1.06 7.99 50.76 0.61 0.19 0.44 0.027

Q CRP1 0.96 8.05 50.57 0.68 0.23 0.49 0.026

Q CRP2 1.26 7.75 54.51 1.87 0.43 1.65 0.135

Q CRP2 1.25 7.75 54.40 1.87 0.46 1.67 0.139

Q CRP2 1.26 7.75 54.12 1.88 0.44 1.63 0.134

Q CRP3 2.17 6.82 9.20 8.80 4.53 5.41 0.354

Q CRP3 2.18 6.81 9.31 8.64 4.46 5.39 0.357

Q CRP3 2.18 6.81 9.36 8.71 4.61 5.26 0.356

Q CRP4 1.10 5.94 22.23 2.08 0.44 1.17 0.056

Q CRP4 1.13 5.91 22.07 2.08 0.44 1.20 0.053

Q CRP4 1.13 5.91 22.04 2.07 0.45 1.18 0.056

R CRP1 0.83 7.84 50.39 0.68 0.21 0.58 0.035

R CRP1 0.84 7.77 50.40 0.67 0.19 0.56 0.033

R CRP1 0.86 7.73 50.95 0.63 0.19 0.57 0.032

R CRP2 0.92 7.56 54.12 1.82 0.39 1.88 0.156

R CRP2 1.16 7.58 54.23 1.82 0.38 1.89 0.155

R CRP2 1.15 7.56 54.53 1.81 0.40 1.85 0.156

R CRP3 1.83 6.68 8.70 8.75 5.40 6.05 0.453

R CRP3 1.75 6.73 8.77 8.55 5.55 6.05 0.449

R CRP3 1.89 6.72 8.23 8.30 5.50 6.35 0.448

R CRP4 1.09 5.85 21.34 2.44 0.57 1.59 0.079

R CRP4 0.92 5.88 21.22 2.58 1.152

2.10 0.095

R CRP4 0.90 5.91 22.02 2.38 0.64 1.70 0.081


Lab Code



S CRP1 1.03 7.88 50.88 0.59 0.18 0.47 0.023

S CRP1 1.02 7.87 52.66 0.59 0.18 0.48 0.022

S CRP1 1.05 7.87 51.59 0.59 0.19 0.48 0.023

S CRP2 1.50 7.70 54.06 1.64 0.39 1.71 0.122

S CRP2 1.50 7.71 54.16 1.65 0.40 1.59 0.130

S CRP2 1.49 7.70 53.64 1.68 0.41 1.65 0.128

S CRP3 2.42 6.80 10.12 6.76 3.19 5.03 0.286

S CRP3 2.47 6.81 9.82 6.87 3.22 5.09 0.294

S CRP3 2.46 6.73 10.06 6.67 3.15 4.77 0.282

S CRP4 1.18 5.73 22.57 1.53 0.34 1.07 0.036

S CRP4 1.23 5.70 22.40 1.61 0.35 1.01 0.037

S CRP4 1.21 5.73 21.80 1.73 0.38 1.07 0.045

1 Eliminated as a Levene’s Test outlier prior to the r&R calculations.

2 Eliminated as an obvious outlier prior to conducting Grubbs’ Test. Reps 1 and 3 were retained.

5. Data – Statistical Analysis

A statistical analysis was conducted in basic conformance with ISO 5725-2:1994 and ISO/TR

22971:2005. A summary of the results from outlier detection and the calculated results for

repeatability (r) and reproducibility (R) are given below in sections 5.1 and 5.2, respectively.

Even though ISO 5725-2:1994 does not suggest calculation of z-scores, z-scores are presented

in section 5.3 so that the participating laboratories would have an additional measure of their

performance compared to their peers. Section 5.4 describes the stability analysis conducted

using data from the 2010-2014 studies. Raw data plots that include all replicates, without

removal of outliers, are given in Appendix A

5.1 Exclusion of Outliers

Procedures outlined in ISO 5725-2:1994 and ISO/TR 22971:2005 were generally used for the

exclusion of outliers. An adaptation of Levene’s Test was used for eliminating laboratories

with overly large repeatability standard deviations and Grubbs’ Test was used to eliminate

laboratories with outlying mean values.

ISO 5725(2) also recommends the use of Mandel’s h and k plots. Mandel’s h statistic is the

same as the statistic used in Grubbs’ Test. Similarly Mandel’s k statistic, associated with

within lab standard deviation, is statistically equivalent to the c-value calculated in Cochran’s

Test ( cnk labs ). However, the critical values associated with Mandel’s h and k statistics do

not make allowance for multiple testing and can therefore, give a false impression of

statistical significance. Thus, Mandels’s h and k statistics do not add fundamentally new

information and may lead to incorrect conclusions. For those reasons, we do not include

Mandel’s h and k plots.

The intent of ISO 5725-2:1994 is to eliminate outliers that exceed a 1% critical value. This

was accomplished by an adaptation of Levene’s Test. Levene’s Test is preferable to

Cochran’s Test, which is recommended in ISO 5725-2:1994, because of Cochran’s Test’s

extreme sensitivity to deviations from normality. Grubbs’ Test and an adaptation of Levene’s

Test were applied at the standard nominal 1% significance level to determine outliers and the

results are shown in Table 4. Levene’s Test is mentioned in ISO/TR 22971:2005 as an


alternative to Cochran’s Test. However, Levene’s Test does not directly apply without

adaptation. For more details, see the footnote below.5

Table 4: Outliers among 2014 Data

Analyte Product Levene’s Outliers Lab Grubbs’ Outliers Lab

pH CRP2 F −

The (-) symbol indicates an outlier was not detected.

As noted above, we do not recommend the use of Cochran’s Test; however we recognize that

it is the approach that has been traditionally employed. So, for comparison, we contrast

Cochran’s Test to Morton’s adaptation of Levene’s Test and show the results in Appendix C.

5.2 Calculation of Repeatability (r) and Reproducibility (R)

After removal of outlying data based on numerical data consistency methods (Grubbs’ Test,

Levene’s Test), the final repeatability and reproducibility (r & R) results were calculated. The

r & R results are shown in Table 5 for the 2014 WG4 study. Since this test was conducted to

evaluate the stability of the CRMs, it should be understood that the r&R results reflect both

laboratory variability and product consistency. That is always true to some degree, but is

particularly true in studies such as this in which the product stability is being evaluated and

may play a non-negligible role in the results.

Table 5: Repeatability (r) and Reproducibility (R) for 2014 WG4 Study

Parameter Product No of

Labs* Mean

Repeatability Reproducibility

r % r R % R

Nicotine (%) CRP1 11 0.95 0.113 11.9% 0.249 26.2%

Nicotine (%) CRP2 11 1.25 0.123 9.8% 0.316 25.3%

Nicotine (%) CRP3 11 2.13 0.085 4.0% 0.445 20.9%

Nicotine (%) CRP4 11 1.08 0.166 15.3% 0.287 26.5%

pH CRP1 11 7.82 0.091 NA 0.576 NA

pH CRP2 10 7.64 0.029 NA 0.290 NA

pH CRP3 11 6.74 0.068 NA 0.262 NA

pH CRP4 11 5.88 0.075 NA 0.252 NA

Moisture (%) CRP1 11 51.2 1.96 3.8% 4.60 9.0%

Moisture (%) CRP2 11 54.2 0.54 1.0% 1.50 2.8%

Moisture (%) CRP3 11 9.10 0.66 7.3% 2.51 27.7%

Moisture (%) CRP4 11 22.0 1.16 5.3% 2.03 9.3%

NNN (µg/g) CRP1 11 0.66 0.087 13.2% 0.143 21.7%

NNN (µg/g) CRP2 11 1.85 0.108 5.8% 0.393 21.3%

5 Levene’s Test is commonly used to determine if each of several subpopulations have the same variance. Since

it was designed to test for overall differences, not to determine if the largest variance is significantly greater than

the others, some adaptation is necessary to use the approach to eliminate laboratories whose within lab variation

is too large. Levene’s test was adapted to this purpose by Morton, who presented the approach utilized in this

report at the 2014 CORESTA Congress (Quebec, Canada, presentation ST28, October 14, 2014). Specifically,

the approach taken here is a two-step process with a lab being eliminated as an outlier if both steps are

statistically significant. First, Levene’s Test was run at a nominal -level of 0.02. Second a comparison of the

largest variance to the remaining variances is carried out at a one-sided nominal level of =0.01/number of labs.

Dividing by the number of labs is to account for multiple testing, since it is not known a priori which lab will

have the largest variance. Simulation studies were carried out by Morton and presented at the 2014 CORESTA

Congress and these results demonstrated that this process has an overall -level near 0.01 and is robust to

deviations from normality.



Labs* Mean


r % r R % R

NNN (µg/g) CRP3 11 8.29 0.741 8.9% 2.698 32.5%

NNN (µg/g) CRP4 11 2.02 0.236 11.7% 0.677 33.4%

NNK (µg/g) CRP1 11 0.20 0.027 13.2% 0.045 21.8%

NNK (µg/g) CRP2 11 0.44 0.036 8.0% 0.098 22.2%

NNK (µg/g) CRP3 11 4.34 0.490 11.3% 1.951 44.9%

NNK (µg/g) CRP4 11 0.46 0.104 22.7% 0.174 37.9%

NAT (µg/g) CRP1 11 0.52 0.054 10.5% 0.137 26.6%

NAT (µg/g) CRP2 11 1.76 0.096 5.4% 0.419 23.8%

NAT (µg/g) CRP3 11 5.76 0.697 12.1% 1.540 26.7%

NAT (µg/g) CRP4 11 1.31 0.267 20.3% 0.578 44.0%

NAB (µg/g) CRP1 11 0.031 0.006 19.8% 0.020 65.8%

NAB (µg/g) CRP2 11 0.149 0.012 7.8% 0.060 40.0%

NAB (µg/g) CRP3 11 0.386 0.039 10.2% 0.198 51.2%

NAB (µg/g) CRP4 11 0.061 0.015 24.8% 0.037 60.4%

*This is the number of laboratory data sets included after removal of outliers.

NA = Since pH is not a proportional scale, it is not appropriate to calculate % r or % R.

5.3 Calculation of Z-Scores

Although calculation of z-scores is not suggested in ISO 5725-2:1994, z-scores were

calculated so that the participating laboratories could compare their results to those of their

peers. It is expected that most of the data should fall within the range of ±2. A final summary

table of z-scores is presented in Table 6 for the 2014 WG4 study. Outliers detected with

Levene’s test were removed prior to the calculation of the z-scores and are given as “Outlier”

in the tables.

Table 6: Z-Scores for 2014 WG4 Study

Product Lab Nicotine pH Moisture NAB NAT NNK NNN

CRP1 A 0.65 0.52 -0.78 1.14 0.94 1.51 0.05

CRP1 F -1.94 -0.19 0.15 0.18 0.08 0.07 0.68

CRP1 G 0.67 0.64 0.85 1.25 1.70 0.69 1.99

CRP1 I 0.08 0.62 0.61 -0.01 0.21 0.74 0.59

CRP1 L 1.01 0.41 -0.26 0.84 -0.55 -0.17 -1.23

CRP1 M 0.49 0.74 -1.96 -0.11 -0.02 0.61 -0.41

CRP1 O -0.92 -1.22 1.97 -2.06 0.10 -0.20 -0.45

CRP1 P -0.36 -2.41 -0.08 0.37 -1.54 -1.74 0.79

CRP1 Q 0.57 0.87 -0.44 -0.76 -1.21 0.52 -0.42

CRP1 R -1.31 -0.21 -0.40 0.32 1.14 -0.36 0.01

CRP1 S 1.05 0.25 0.34 -1.16 -0.84 -1.67 -1.59

CRP2 A 0.37 0.25 1.14 1.12 1.12 1.29 -0.26

CRP2 F -0.68 Outlier1

0.48 -0.34 0.06 0.44 0.11

CRP2 G 0.04 0.64 0.82 1.85 1.67 1.08 1.76

CRP2 I 0.00 0.35 -1.32 0.45 0.51 0.96 0.94

CRP2 L 0.29 0.48 0.53 -0.04 -0.45 -0.20 -0.77

CRP2 M 0.56 0.61 0.45 0.02 -0.03 0.33 -0.47

CRP2 O -1.00 -1.49 0.37 -1.82 -0.11 0.22 -1.22


Product Lab Nicotine pH Moisture NAB NAT NNK NNN

CRP2 P -0.34 -1.88 -2.26 0.20 -1.94 -1.37 1.29

CRP2 Q 0.04 1.09 0.21 -0.64 -0.78 0.06 0.20

CRP2 R -1.60 -0.68 0.12 0.28 0.72 -1.49 -0.21

CRP2 S 2.31 0.64 -0.54 -1.08 -0.77 -1.32 -1.38

CRP3 A 0.39 -1.16 0.20 0.18 1.33 0.54 0.24

CRP3 F 0.11 -2.08 -0.22 -0.20 0.24 0.71 0.31

CRP3 G 0.16 -0.25 -0.15 1.59 1.47 -0.58 1.22

CRP3 I 0.00 1.10 -1.79 0.16 -0.40 -0.01 0.48

CRP3 L 0.49 -0.34 -0.64 0.22 0.07 -0.15 0.34

CRP3 M 0.15 1.03 1.38 0.27 -0.12 -0.34 0.06

CRP3 O -1.12 -0.11 1.39 -1.94 -1.42 -1.42 -2.21

CRP3 P -0.54 0.99 -0.95 0.67 0.42 0.99 0.46

CRP3 Q 0.29 0.77 0.26 -0.44 -0.79 0.28 0.45

CRP3 R -1.96 -0.36 -0.56 0.91 0.77 1.67 0.26

CRP3 S 2.02 0.41 1.08 -1.42 -1.56 -1.69 -1.62

CRP4 A -0.12 0.22 0.64 0.65 0.17 0.07 0.08

CRP4 F -1.38 -2.11 0.98 -0.56 -0.59 -0.14 -0.59

CRP4 G -0.13 0.49 -1.18 0.83 0.21 -0.37 0.91

CRP4 I 0.92 0.18 0.01 -0.04 0.13 0.55 0.57

CRP4 L -0.11 0.60 2.01 0.17 -0.68 -0.43 -0.39

CRP4 M 0.02 0.41 -0.34 -0.25 -0.37 -0.66 -0.81

CRP4 O -1.23 0.91 -0.60 -1.07 0.33 0.20 -0.79

CRP4 P 1.51 0.60 -1.42 0.65 0.35 0.28 0.64

CRP4 Q 0.41 0.49 0.20 -0.52 -0.69 -0.29 0.21

CRP4 R -1.22 0.03 -0.72 1.89 2.52 2.44 1.90

CRP4 S 1.34 -1.81 0.42 -1.75 -1.38 -1.67 -1.73

1. Outliers were determined as described in Section 5 and were removed prior to the calculation of the z-

scores.

5.4 CRP Stability Assessment

The data were evaluated for stability by examining the results from the 2010-2014 WG4

studies for trends over time, for all analytes. Even though the stability trend analysis,

described below, was the primary evaluation of stability, the data were also graphically

evaluated for changes in between-lab variability over the study years. As mentioned in the

introduction, the 2010 study did not specify methods of analysis and the 2011 study was the

first study year initiated to assess stability of the CRPs where methods were specified.

However, after evaluating the data for all five years, it was determined that the 2010 data

were consistent with the data from later years and were included in the stability analysis.

The stability trend analysis included two factors: year of analysis and laboratory. To account

for potential year-to-year correlation in a laboratory’s results, the laboratory variable was

treated as a random factor.

There is also a potential need to account for multiplicity of testing. That is, if four statistical

tests are each carried out with a 5% chance to falsely declare a time trend, then there is just

under a 20% chance that at least one of them will give a false indication of a time trend even

in the ideal case of absolute stability. In this instance there are four reference products and

seven separate analytes, thus twenty-eight comparisons in all. The approach taken was to

correct for testing multiplicity across products for each analyte and not across all analytes.


The correction employed was to allow for four tests being conducted not all twenty-eight.

That meant that each test was judged statistically significant if the associated p-value was less

than 5%/4 or 1.25%. Therefore, there is roughly a 5% chance that each analyte will be

declared to show a time trend on one or more of the CRPs by chance alone.

Graphs of the means for all years and all analytes are shown in Appendix B. These graphs

include the complete data set, without removal of outliers, to give the reader a perspective

both on the retained points and those that were excluded as outliers. Table 7 summarizes the

trend analysis of each analyte and each product. Since the testing was done on the mean

values, Grubbs’ Test outliers were identified and removed prior to analysis, but no outliers

were removed for excessive intra-laboratory variation. There were not any statistically

significant trends in the data, though it should be noted that in most years, for most analytes,

there is quite a lot of lab-to-lab variation. This limits the ability to detect statistically

significant trends.

The previous year’s analysis6 showed a slight upward trend in moisture for CRP3. This year

that trend fell just short of statistical significance. The reason for this appears to be because

the 2014 values are generally comparable to 2013 instead of continuing the upward trend.

There is still (on average) statistically higher CRP3 moistures in 2013 and 2014 than in 2010

and 2011.

Table 7: Summary of Stability Analysis

Product Variable Estimated Slope p-value1

CRP1 Nicotine -0.0014 81.0%




CRP1 pH -0.0273 7.0%

CRP2 pH -0.0149 7.7%

CRP3 pH -0.0124 8.4%

CRP4 pH -0.0062 54.5%

CRP1 Moisture2

-0.2948 5.6%

CRP2 Moisture -0.0267 58.4%

CRP3 Moisture 0.1914 1.7%

CRP4 Moisture 0.0774 73.4%

CRP1 NNN -0.0025 81.3%

CRP2 NNN -0.0060 77.3%

CRP3 NNN 0.0632 45.0%

CRP4 NNN -0.0006 98.3%

CRP1 NNK3

-0.0043 1.6%

CRP2 NNK -0.0032 38.6%

CRP3 NNK4

0.0849 8.8%

CRP4 NNK 0.0043 51.9%

CRP1 NAT -0.0073 29.0%

CRP2 NAT -0.0266 24.8%

CRP3 NAT -0.0115 87.7%

CRP4 NAT 0.0104 61.7%

CRP1 NAB4

0.00048 61.0%

6 STS Technical Report: CORESTA Reference Products 2012 and 2013 Analysis, July 2014.


Product Variable Estimated Slope p-value1

CRP2 NAB -0.00093 63.1%

CRP3 NAB -0.00145 78.2%

CRP4 NAB4

-0.00013 88.7%

1. P-values are declared statistically significant if p < 1.25%. 1.25% is derived by dividing the

nominal 5% level by 4 to account for testing multiplicity per analyte.

2. Calculated omitting the Lab E result in 2010.

3. Calculated omitting the Lab P result in 2013.

4. Calculated omitting the Lab K result in 2012.

6. Data Interpretations

Though the trend of increased average moisture content noted in last year’s report for CRP3 is

no longer statistically significant, the current average moisture values are still somewhat

higher than those shortly after the product was manufactured. Still this increase was marginal

and equated to an increase of approximately 1% when calculated as an average for all labs.

CRP3 has the lowest moisture content so this product would be most likely to absorb water

over time as compared to the other CRPs. As mentioned earlier, results are presented on an

as-is basis; however, this negligible shift in moisture for CRP3 would not be evident in the

other analyses considering the negligible change in moisture in comparison with lab-to-lab

method variability.

In examining the mean data plots for all years (Appendix B), a number of potential outliers

are evident; therefore, the stability analyses were conducted both with and without the outliers

included, but only shown in the table above with the outliers excluded. There were some

indications of changes in between-lab variability over the study years. Two notable examples

of increased variability between labs are pH for CRP3 and moisture for CRP1. Both have

increased in variability across the reporting period (2010 Lab E was dropped as an outlier for

CRP1).

As reported last year, the inter-laboratory variability for CRP4 moisture has been very high in

some years, such as 2012, but was improved for 2014. This year, one laboratory reported that

a package of CRP4 that was recently received from NCSU appeared to be uncharacteristically

dry. The laboratory confirmed that the moisture content was approximately half of what it

should have been and retested another package (all analyses). Although the original package

of CRP4 was clearly defective a smaller loss of moisture would not have been readily

apparent and any differences in the moisture content determined would have likely been

attributed to method variability.

Due to the product type, CRP4 is the least homogenous of the four CRPs. This suggests that

sample preparation (particle size reduction) and how tobacco aliquots are removed for testing

may be important components of the variability seen within labs for CRP4. Labs reported a

range of techniques ranging from cryogrinding an entire package to simply removing

individual aliquots for testing. Variability may potentially be reduced by generating a

separate homogeneous composite sample for each replicate and removing the aliquot for the

replicate from that homogeneous composite. This could be accomplished by either

cryogrinding an entire package or using a razor blade to cut and mix the entire contents of a

package before removing the sample for analysis. We recommend this approach in the future.

Overall, both nicotine and TSNAs are shown to be stable over the five study years, for all

CRPs. This is not unexpected as both nicotine and TSNAs should be stable in these products

when the products are maintained at −20°C.


Based on the results taken in their entirety, the conclusion of this study is that all four CRPs

are suitable for continued use as reference products. Lack of any clear and significant trends

in the stability results indicate that storage at −20°C is an appropriate storage condition for the

four reference products. Furthermore, since the changes in moisture, if any, are small relative

to the inherent method reproducibilities, it is unlikely that they will have an easily detectable

effect on other results.

7. Recommendations

The Smokeless Tobacco Sub-Group recommends that WG4 continue to monitor stability of

the CRPs on an annual basis. As stated earlier, data generated in 2010 will serve as the

baseline for future WG4 stability analyses. A formal analysis of product stability should be

conducted with data obtained from the next WG4 study in the spring of 2015. For future

analyses, we are recommending that all CRPs be obtained from NC State University just prior

to the study to mitigate differences in long term storage conditions by the participating

laboratories. Furthermore, we are recommending that CRP4 be prepared for analysis by

either cryogrinding an entire package or by finely cutting and mixing an entire package with a

razor blade before removing an aliquot for analysis.


APPENDIX A: Raw Data Plots for 2014 WG4 Study


APPENDIX B: CRP Stability Assessment, Mean Data Plots


APPENDIX C: Results Using Cochran’s Test for Outlier Detection

As discussed above, we do not recommend the use of Cochran’s Test; however we recognize that it is

the approach that has been traditionally employed. So, for comparison, we contrast Cochran’s Test to

Morton’s adaptation of Levene’s Test and show the Results in APPENDIX B.

The outliers identified when Cochran’s Test is employed are given in the following table. Note that

the Grubbs’ Test outliers show slight differences as well because Grubbs’ Test was applied after the

removal of intra-lab variation outliers.

Outliers among 2014 data using Cochran’s Test followed by Grubbs’ Test (for comparison

purposes only as this is not the recommended approach)

Analyte Product Cochran’s Outliers -

Lab Grubbs’ Outliers -

Lab

Nicotine CRP2 R,P −

Nicotine CRP4 F,R −

pH CRP2 F,P −

pH CRP4 F S

NAB CRP3 O −

NAT CRP3 O −

NAT CRP4 R −

NNN CRP2 F −

NNN CRP3 O S

Moisture CRP3 A −

Moisture CRP4 F −

The corresponding r&R table is given below when using Cochran’s Test followed by

Grubbs’ Test (for comparison purposes only as this is not the recommended approach).


Labs* Mean


r % r R % R

Nicotine (%) CRP1 11 0.95 0.113 11.9% 0.249 26.2%

Nicotine (%) CRP2 9 1.27 0.035 2.7% 0.280 22.0%

Nicotine (%) CRP3 11 2.13 0.085 4.0% 0.445 20.9%

Nicotine (%) CRP4 9 1.11 0.064 5.8% 0.223 20.1%

pH CRP1 11 7.82 0.091 NA 0.576 NA

pH CRP2 9 7.66 0.020 NA 0.231 NA

pH CRP3 11 6.74 0.068 NA 0.262 NA

pH CRP4 9 5.92 0.044 NA 0.074 NA

Moisture (%) CRP1 11 51.2 1.96 3.8% 4.60 9.0%

Moisture (%) CRP2 11 54.2 0.54 1.0% 1.50 2.8%

Moisture (%) CRP3 10 9.0 0.43 4.7% 2.60 28.8%

Moisture (%) CRP4 10 21.9 0.73 3.3% 1.89 8.6%

NNN (µg/g) CRP1 11 0.66 0.087 13.2% 0.143 21.7%

NNN (µg/g) CRP2 10 1.84 0.078 4.2% 0.408 22.1%

NNN (µg/g) CRP3 9 8.69 0.522 6.0% 0.960 11.1%

NNN (µg/g) CRP4 11 2.02 0.236 11.7% 0.677 33.4%

NNK (µg/g) CRP1 11 0.20 0.027 13.2% 0.045 21.8%

NNK (µg/g) CRP2 11 0.44 0.036 8.0% 0.098 22.2%

NNK (µg/g) CRP3 11 4.34 0.490 11.3% 1.951 44.9%



Labs* Mean


r % r R % R

NNK (µg/g) CRP4 11 0.46 0.104 22.7% 0.174 37.9%

NAT (µg/g) CRP1 11 0.52 0.054 10.5% 0.137 26.6%

NAT (µg/g) CRP2 11 1.76 0.096 5.4% 0.419 23.8%

NAT (µg/g) CRP3 10 5.83 0.498 8.5% 1.391 23.9%

NAT (µg/g) CRP4 10 1.27 0.147 11.6% 0.333 26.3%

NAB (µg/g) CRP1 11 0.03 0.006 19.8% 0.020 65.8%

NAB (µg/g) CRP2 11 0.15 0.012 7.8% 0.060 40.0%

NAB (µg/g) CRP3 10 0.40 0.024 6.1% 0.159 39.7%

NAB (µg/g) CRP4 11 0.06 0.015 24.8% 0.037 60.4%

*This is the number of laboratory data sets included after removal of outliers.

NA = Since pH is not a proportional scale, it is not appropriate to calculate % r or % R.

As noted by Morton in a 2014 presentation at the CORESTA Congress, Cochran’s Test tends to

identify more outliers than the adaptation of Levene’s Test. As a general rule, the calculated

repeatability tends to be lower when Cochran’s Test is employed. The estimated reproducibility tends

to be similar with the two approaches, but in some cases can be quite different as illustrated here. For

example, in examining NNN in CRP3, after Lab O is dropped as a Cochran’s Test outlier, then Lab S

is identified as a Grubbs’ Test outlier and the reproducibility is much less than with both of those Labs

included in the analysis.

smokeless tobacco sub-group - coresta · smokeless tobacco sub-group ... karl wagner, ph.d. altria...

Documents