ucrl-vg-137402 this work was performed under the auspices of the u.s. department of energy by...
TRANSCRIPT
2011 LLNL Template
The Importance of Having All Data Users Know the Analytical Data Quality Objectives: Lessons LearnedDOE Analytical Services Program 2014 WorkshopDon MacQueenUCRL-VG-137402
This work was performed under the auspices of the U.S. Department of Energy by Lawrence Livermore National Laboratory under contract W-7405-Eng-48.Lawrence Livermore National LaboratoryUCRL-VG-134702#1Once upon a time...There was a site
At which three organizations decided to collect some samples for a radiological assessment
And split them
But the results didnt come out quite as well as they had hoped...Lawrence Livermore National LaboratoryUCRL-VG-134702#1995 Results
Lawrence Livermore National LaboratoryUCRL-VG-134702#1995: 19 Soil SamplesSplit in the field between three organizations
Lawrence Livermore National LaboratoryUCRL-VG-134702#Analyzed for 239+240Pu, 238PuKey questions for 239+240Pu:
Is it above 2.5 pCi/g?i.e., above a derived-from-risk screening level?
Is it above 0.01 pCi/g?i.e., is it above fallout background?More precisely, is it above a value being used to represent an upper limit for fallout background?
How did the Pu reach the site?Lawrence Livermore National LaboratoryUCRL-VG-134702#Method and resultsTotal dissolutionChemical separationAlpha spectroscopy
Lab A and B results were surprisingly differentLab A and C good agreementLawrence Livermore National LaboratoryUCRL-VG-134702#Locations 4 and 13
Lawrence Livermore National LaboratoryUCRL-VG-134702#Locations 3 and 9
Lawrence Livermore National LaboratoryUCRL-VG-134702#Locations 7 and 8
Lawrence Livermore National LaboratoryUCRL-VG-134702#Why are A and B so different?Data are obviously of different quality
Organization A didnt really understand whyNoted that counting times were differentSpeculated about particulate nature of radionuclides in soil
Noted that some Lab B QC showed large variation
Organization A wrote a report, used uncertainty-weighted average of Lab A and B resultsLawrence Livermore National LaboratoryUCRL-VG-134702#1998: Organization DWas not involved in any way in 1995
Reviewed data, released a draft report for public comment
Used only Lab B resultsCounted number of samples above 0.01 pCi/g
Reached a very different conclusion about how the Pu reached the siteLawrence Livermore National LaboratoryUCRL-VG-134702#Alternate InterpretationsBased on Lab B results13 of 19 above 0.01 pCi/g, ignoring uncertainty6 of 19 above 0.01 pCi/g, considering uncertainty13/19 = 68% = throughout the site = air pathway
Based on Lab A results6 of 19 above 0.01 pCi/g, ignoring uncertainty6 of 19 above 0.01 pCi/g, considering uncertainty5 of the 6 located consistent with sludge pathway Lawrence Livermore National LaboratoryUCRL-VG-134702#Locations 4 and 13
Lawrence Livermore National LaboratoryUCRL-VG-134702#Locations 3 and 9
Lawrence Livermore National LaboratoryUCRL-VG-134702#Details from two samples
Lawrence Livermore National LaboratoryUCRL-VG-134702#Counts to exceed 0.01 pCi/gIn order to exceed 0.01 pCi/g (the so-called background)Lab A needed 20 to 30 net countsLab B needed 2 to 4 net counts
Is it valid to use Lab B results to make above/below background inferences?Not alwaysLab B results are much more variable at low levelsExample: 0.0105 0.0189 with MDC = 0.0438 was considered evidence of being above 0.01 pCi/g by Organization DLawrence Livermore National LaboratoryUCRL-VG-134702#Data quality objectivesInstructions to Lab A:MDC should be 0.005 pCi/g or lessThis is a surrogate for good performance at low levels
Instructions to Lab B:Tell us if it is above 2.5 pCi/gThis implies that large variability at low levels is acceptableLawrence Livermore National LaboratoryUCRL-VG-134702#Lab implementation of DQOsLab A5 gram aliquot24 hour counting time
Lab B0.5 gram aliquot16 2/3 hours counting time
This explains why the two labs have such different results on many samples, and why Lab B results are so variable
Aliquot size is the more crucial of these Lawrence Livermore National LaboratoryUCRL-VG-134702#Particulate Sampling
From Pitard, F., 1993 as reproduced in Myers, J., 1997Don MacQueen:Note, demonstrate concept visually here using bag of granola, with raisins as the contaminantLawrence Livermore National LaboratoryUCRL-VG-134702#What went wrongLab Bs results do not measure small differences near background precisely, but this was not well understoodMost especially, by Organization D!Based on their draft report, Organization D did not attempt to understand DQOs, or differences between data setsOrganization A did not understand well enough soon enough to effectively discuss Ds interpretationOrganizations A, B, and C had little or no discussion of DQOsOrganizations B and C did not raise the DQO issue when it would have helpedOrganization As 1995 report would have been better if they had understood the DQOs more thoroughlyLawrence Livermore National LaboratoryUCRL-VG-134702#What about the analytical labs?All of the labs met their clients requirements
It is the clients responsibility to understand and share DQOs
But, Lab B could have helpedLab B was aware that comparisons with 0.01 pCi/g were of interestLab B project manager did make comparisons with 0.01 pCi/g in the analytical report cover letter
Lawrence Livermore National LaboratoryUCRL-VG-134702#First lessonWhen splitting samples, clients should insist on the same data quality objectives. Organization A did not do this.
If the DQOs are not the same, make sure everyone understands the limitations (and document them).Lawrence Livermore National LaboratoryUCRL-VG-134702#
We learned our lesson: 1998Lawrence Livermore National LaboratoryUCRL-VG-134702#SummaryData users were not sufficiently aware of the 1995 data quality objectives
Data users did not understand the limitations of Lab Bs 1995 results (limits that were specified by Organization B and met their needs)
Misuse of Lab Bs results enabled Organization D to introduce a huge red herring into the site assessment processBecause of failure to consider the data quality objectivesLawrence Livermore National LaboratoryUCRL-VG-134702#Lessons learnedThe clients responsibility does not end at the labs front door
Good scientific interpretation of a number requires full knowledge of how the number was generated
Coordinate, Cooperate, COMMUNICATE!
Lawrence Livermore National LaboratoryUCRL-VG-134702#OrganizationA(LLNL)BC
LabA(not LLNL)BC
Number of samples19197
Net countsBackground CountsResult
A210.0007 0.0014
B2.3350.0105 0.0189
A1110.0036 0.0023
B4.3320.0111 0.0115