Christian C. Matchett, F-ABCGeorgia Bureau of Investigation

presented at 2008 CLIC Technical Training Seminar

2007/2008 SWGDRUG Updates 2007/2008 SWGDRUG Updates Uncertainty DocumentUncertainty Document

Drug Enforcement AdministrationDrug Enforcement AdministrationOffice of Forensic SciencesOffice of Forensic Sciences

sponsored by the

National Institute of StandardsNational Institute of Standardsand Technologyand Technology

and the

2007/2008 SWGDRUG Updates 2007/2008 SWGDRUG Updates Uncertainty DocumentUncertainty Document

Why Address Uncertainty?Why Address Uncertainty?

Forensic community asking for guidance Accrediting bodies establishing measures of

assessing conformity with ISO Customer requirements Jurisdictional requirements

Transparency (nothing to hide) Potential Exculpatory Information

Core Committee Actions

Jan 2008 – Draft Uncertainty document approved and released for public comment

Posted on the website since February 2008 July 2008 – Comments were addressed and final

document adopted on July 22, 2008 The document should be posted on the website

by the end of September 2008

SWGDRUG ApproachSWGDRUG Approach

Tailor the recommendations to drug analysis and answer specific uncertainty questions

Ensure dialog on topic is not dictated to us by those outside of the seized drug community

Offer guidance and direction to laboratories and accrediting bodies

Aware that there is a wealth of information that already exists on Uncertainty

No intentions of repeating existing information

Purpose of DocumentPurpose of Document Provide General Guidance

Principles and themes conveyed, not step-by-step instructions

Raise Awareness and Put In Context Uncertainty is not doubt, it provides assurance

that results and conclusions are fit for purpose Determine Laboratory Responsibility

Consider customer requirements and address uncertainty through training, procedures and documentation

State Benefits Enhanced confidence through increased

understanding of results Provides Mechanism to express reliability of

results

Application of Application of UncertaintyUncertainty

Qualitative Quantitative

Purity Weights

Qualitative AnalysisQualitative Analysis

Individual methods have limitations and, consequently, uncertainty

Understanding limitations allows analysts to build an appropriate analytical scheme to correctly ID drugs or chemicals

It is expected that an appropriate analytical scheme will result in, effectively, no uncertainty in reported identifications

Relevant limitations should be documented and may need to be in report

Qualitative ExamplesQualitative Examples

Use Part III B Methods of Analysis/Drug Identification

IR and microcrystalline test positive for cocaine – effectively NO uncertainty

Limitations Marquis test positive for methamphetamine –

could be methamphetamine or other amphetamines

GC/MS test positive for ephedrine – could be ephedrine or pseudoephedrine

Quantitative AnalysisQuantitative Analysis

Uncertainty is defined as an estimate attached to a test result which characterizes the range of values within which the true value is asserted to lie

Precise calculations of measurement uncertainty is not always required

Seized Drug Numerical ResultsSeized Drug Numerical Results

Primary numerical values reported in the analysis of seized drugs are

Weight and Purity

Where a value is critical, an appropriate measurement uncertainty determination shall be applied

Weight close to a statutory threshold Purity of drug close to level which affects

sentencing

WeightWeight

Uncertainty of a reported value is dependant on the weighing process. Factors include:

Single item versus multiple items (# of weighing operations)

Tare function as separate weighing operation Extrapolation of population weight from limited

sampling of multiple items Aggregate weighings Incomplete recovery of material from packaging Balance selection (e.g., readability, capacity) Balance operation (e.g., sample placement,

environmental conditions)

PurityPurity

Sources of uncertainty for purity determination

Sampling plan (e.g., handling of multiple exhibits) Sample homogeneity

Analytical method Sample preparation (e.g., size, matrix effects,

solubility) Analytical technique Reference material (e.g., purity of standard) Equipment and instrumentation performance (e.g.,

glassware, pipetters, balances, chromatographs) Concentration of analyte Environmental conditions

Purity ApproachesPurity Approaches Analytical Error

Address both systematic and random error through method validation and quality assurance

Sampling Error The sample and sampling procedure are often

the greatest contributors to measurement uncertainty

Where appropriate, confidence levels (e.g., 95%, 99.7%) shall be selected based on considerations relevant to the analytical context

Record uncertainty information in validation documents and/or case records

Uncertainty BudgetUncertainty Budget

All sources of error are separately identified and tabulated

Assign values to each error source using Empirical data

Validation process, Historical performance data, Control chart data, proficiency tests

Published data Combination of empirical and published data

Can exclude insignificant sources Calculate combined and expanded uncertainty

using significant values for procedure

Non-Budget ApproachNon-Budget Approach

Example 1: Use of data from replicate analyses from a validated method with an appropriate sampling plan

Sources of uncertainty that are separately assessed in the budget method are collectively assessed by experimental measurements

Example 2: Use of two standard deviations (2σ) of the test method results from reproducibility data from the validation studies.

Provides an approximation of the measurement uncertainty for non-critical values

ReportingReporting

Uncertainty shall be documented but may not need to be reported Should be reported when

result impacts customer Even if not reported,

analysts shall be cognizant of the uncertainty associated with their results

When to Report Uncertainty?When to Report Uncertainty?

Jurisdictional Prevailing statutory requirement Relevant governing body (agency) requirements Customer requests Potential exculpatory value

Analytical Qualitative results where limitations are known (e.g.,

inability to differentiate isomers) Quantitative measurements where a value is critical

(e.g., weight or purity level close to statutory threshold)

Laboratory accreditation requirements

Qualitative Reporting ExamplesQualitative Reporting Examples

Contains Ephedrine or Pseudoephedrine Item tested: 5.2 grams net

Visual examination determined that the physical characteristics are consistent with a Schedule IV pharmaceutical preparation containing Diazepam. There was no apparent tampering of the dosage unit and no further tests are being conducted.

Contains cocaine (salt form not determined)

Quantitative Reporting ExamplesQuantitative Reporting Examples

Positive for cocaine in the sample tested Net weight of total sample: 5.23 grams ± 0.03 grams Quantitation: 54.7% ± 2.8%

Sample tested positive for cocaine Net weight: 5.23 grams Purity: 54.7% Confidence Range: ± 2.8%* Calculated net weight of drug: 2.8 grams of cocaine *Confidence range refers to a 95% confidence level

Training RecommendationsTraining Recommendations

Individuals responsible for determining, evaluating and documenting uncertainty shall be capable of demonstrating familiarity with foundational concepts and principles of estimating uncertainty

General metrology (terminology, symbols, etc.) Concepts of random and systematic error, accuracy,

precision, propagation of error, etc. Reporting conventions (sig. figs, truncating, rounding) Basic statistics (i.e., confidence interval, probability,

etc) All analysts shall be capable of explaining their

labs procedures for evaluating uncertainty of qualitative and quantitative analyses

Supplemental Documents to Supplemental Documents to FollowFollow

Uncertainty budget Control chart data applications Demonstration of balance control using

standard weight sets Summing weights from individual exhibits Expression of sampling uncertainty based on

confidence interval using multiple samplings

Core CommitteeCore Committee

• DEA – Nelson Santos DEA – Nelson Santos (Chair)(Chair)

• Secretariat – Scott Secretariat – Scott Oulton (non-voting)Oulton (non-voting)

• FBI - Eileen WaningerFBI - Eileen Waninger

• ASCLD – Garth ASCLD – Garth GlassburgGlassburg

• NIST - Susan BallouNIST - Susan Ballou

• ASTM and NEAFS- ASTM and NEAFS- Jack MarioJack Mario

• CAC & NWAFS - Jerry MassettiCAC & NWAFS - Jerry Massetti

• MAFS - Richard PaulasMAFS - Richard Paulas

• MAAFS - Linda JacksonMAAFS - Linda Jackson

• SAFS – Christian MatchettSAFS – Christian Matchett

• Toxicology – Dr. Robert PowersToxicology – Dr. Robert Powers

• Educator – Dr. Chris TindallEducator – Dr. Chris Tindall

• Educator – Dr. Suzanne BellEducator – Dr. Suzanne Bell

Core CommitteeCore Committee

Core CommitteeCore Committee

• Canada - Richard LaingCanada - Richard Laing

• Japan – Mr. Osamu Japan – Mr. Osamu OhtsuruOhtsuru

• United Kingdom - Dr. United Kingdom - Dr. Sylvia BurnsSylvia Burns

• Australia - Catherine Australia - Catherine QuinnQuinn

• Germany - Dr. Udo ZerellGermany - Dr. Udo Zerell

• ENFSI - Dr. Michael ENFSI - Dr. Michael BovensBovens

• UNODC - Dr. Iphigenia UNODC - Dr. Iphigenia NaidisNaidis

THANK YOU

Visit us atVisit us atwww.swgdrug.orgwww.swgdrug.org