Interagency Monitoring of Interagency Monitoring of Protected Visual Protected Visual

Environments (IMPROVE) Environments (IMPROVE) Data PrecisionData Precision

Nicole P. Hyslop and Warren H. Nicole P. Hyslop and Warren H. WhiteWhite

IMPROVE Collocated IMPROVE Collocated (Duplicate) Measurements(Duplicate) Measurements• Established for the first time in 2004.Established for the first time in 2004.• Seven collocated modules of each type. Seven collocated modules of each type. • Two objectivesTwo objectives

– Evaluate precision of measurements andEvaluate precision of measurements and– Determine if uncertainty estimates are accurate.Determine if uncertainty estimates are accurate.

• Captures uncertainty resulting from sample Captures uncertainty resulting from sample collection, sample analysis, and data collection, sample analysis, and data processing.processing.

• Does not capture uncertainty resulting from Does not capture uncertainty resulting from temperature uncertainty or calibration temperature uncertainty or calibration uncertainty.uncertainty.

Module Type

Site Code

Site NameState

Start Date

A PM2.5

mass & elements

MEVE Mesa Verde NP CO 8/13/03

PMRFProctor Maple Research Facility

VT 9/3/03

OLYM Olympic NP WA 11/8/03

PHOE Phoenix AZ 3/30/04

SAFO Sac and Fox Nation KS11/20/03

TRCR Trapper Creek AK 6/22/04

SAMA Saint Marks FL11/18/04

B PM2.5

anions

LAVO Lassen Volcanic NP CA 4/18/03

MACA Mammoth Cave NP KY 5/12/03

BIBE Big Bend NP TX 8/30/03

GAMOGates of the Mountains WA

MT 9/23/03

FRRE Frostburg Reservoir MD 4/15/04

BLMOBlue Mounds State Park

MN 9/16/04

PHOE Phoenix AZ 3/30/04

Module Type

Site Code

Site NameState

Start Date

CPM2.5

organic&

elemental carbon

EVER Everglades NP FL 7/11/03

SENE Seney NWR MI 8/10/03

HOOV Hoover WA CA 8/13/03

MELA Medicine Lake NWR MT 9/25/03

PHOE Phoenix AZ 3/30/04

SAWE Saguaro NP West AZ 3/25/04

HEGL Hercules-Glade WA MO 9/15/04

DPM10

mass

JOSH Joshua Tree NP CA 8/7/03

QURE Quabbin Reservoir MA 9/4/03

HOUS Houston TX 4/4/03

JARB Jarbridge WA NV 6/30/04

PHOE Phoenix AZ 3/30/04

WICA Wind Cave NP SD 9/17/04

SWAN Swanquarter WA NC 11/9/04

Uncertainty and Precision in Uncertainty and Precision in this Presentationthis Presentation

• Uncertainty refers to the expected Uncertainty refers to the expected variance in outcomes of measurements variance in outcomes of measurements repeated under the same conditions.repeated under the same conditions.

• Collocated precision refers to the Collocated precision refers to the variance of relative differences between variance of relative differences between the duplicate measurements. the duplicate measurements.

Collocated Sulfur DataCollocated Sulfur DataSulfur

0.0010

0.0100

0.1000

1.0000

Average Concentration (ug m-3)

-0.4

-0.2

0.0

0.2

0.4

Ari

thm

et ic

Di ff

ere

nc e

(u

g m

-3)

TRCRSAFOPMRFPHOEOLYMMEVEMDL

SITES

0.0010

0.0100

0.1000

1.0000

Average Concentration (ug m-3)

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

Re

lati v

e D

iffe

ren

c e

Collocated Selenium DataCollocated Selenium DataSelenium

0.00001 0.00010 0.00100Average Concentration (ug m-3)

-0.00030

-0.00015

0.00000

0.00015

0.00030

Arit

hmet

ic D

if fer

ence

(ug

m- 3

)

TRCRSAFOPMRFPHOEOLYMMEVEMDL

SITES

0.00001 0.00010 0.00100Average Concentration (ug m-3)

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

Rel

ativ

e D

i ffer

ence

Collocated Precision Collocated Precision CalculationsCalculations

• U.S. EPA designated method (FRM, U.S. EPA designated method (FRM, 1997)1997)

• Only use pairs where the average Only use pairs where the average concentration is greater than 3 * MDL.concentration is greater than 3 * MDL.

pairs sample ofnumber isn

ion,concentrat collocated denotes

ion,concentrat routine denotes

%100*2/

2/1Precision Collocated

1

2

i

i

n

i ii

ii

Y

Y

XY

XY

n

Estimated Uncertainty for each Estimated Uncertainty for each ConcentrationConcentration

• An uncertainty estimate is reported with An uncertainty estimate is reported with every species concentration.every species concentration.– Estimates are based on individual sources of Estimates are based on individual sources of

measurement uncertainty and propagation of measurement uncertainty and propagation of errors.errors.

yuncertaint analyticalconstant

yuncertaint volumealproportion

yuncertaint analytical alproportion

ionconcentrat][ where

constant

2constant

222

Unc

unc

unc

C

UncuncuncCUnc

volume

analytic

volumeanalyticC

Why don’t the collocated Why don’t the collocated precisions meet the precisions meet the expectations?expectations?• Analytical issuesAnalytical issues

– Analysis areaAnalysis area– Minimum detectable limits (mdl’s)Minimum detectable limits (mdl’s)– Concentrations always close to mdlConcentrations always close to mdl

• Sampling discrepanciesSampling discrepancies– Cyclone cutpoint differencesCyclone cutpoint differences– Non-uniform depositsNon-uniform deposits– Deposit areaDeposit area

Copper-anode XRF Element Copper-anode XRF Element MetricsMetricsSpecies

Count > 3*MDL†

Estimated Uncertainty

Collocated Precision

Collocated/ Estimated

Na 15 25% 50% 2.0Mg 9 n < 10 n < 10 --Al 257 14% 69% 5.0Si 533 11% 41% 3.7P 21 18% 131% 7.3S 577 5% 8% 1.6Cl 105 12% 68% 5.6K 577 6% 13% 2.3Ca 577 6% 19% 3.0Ti 528 11% 30% 2.6V 457 18% 21% 1.2Cr 108 28% 56% 2.0Mn 529 12% 24% 2.0Fe 577 5% 18% 3.3

† 577 valid pairs

Sulfur by XRF and Sulfate by Sulfur by XRF and Sulfate by ICIC

SpeciesCount > 3*MDL

Estimated Uncertainty

Collocated Precision

Collocated/ Estimated

Sulfur 577/577 5.3% 8.2% 1.56

Sulfate 801/823 4.8% 4.1% 0.86

Sulfate

0.01 0.10 1.00 10.00Average Concentration (ug m-3)

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

Rel

a tiv

e D

iffe r

ence

PHOE1MACA1LAVO1GAMO1FRRE1BLMO1BIBE1AVG_MDL

SITES

Sulfur

0.0010

0.0100

0.1000

1.0000

Average Concentration (ug m-3)

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

Re

lativ

e D

iffe

r en

ce

TRCRSAFOPMRFPHOEOLYMMEVEMDL

SITES

Soil-derived Elements Soil-derived Elements Calcium

0.0001 0.0010 0.0100 0.1000Average Concentration (ug m-3)

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

Re

lativ

e D

if fe

r en

ce

TRCRSAFOPMRFPHOEOLYMMEVEMDL

SITES

Iron

0.00010

0.00100

0.01000

0.10000

Average Concentration (ug m-3)

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

Re

lativ

e D

iffe

r en

ce

TRCRSAFOPMRFPHOEOLYMMEVEMDL

SITES

SpeciesCount > 3*MDL

Estimated Uncertainty

Collocated Precision

Collocated/ Estimated

Ca 577/577 6.2% 18.8% 3.0Fe 577/577 5.5% 17.9% 3.3

0

0.5

1

1.5

2

0 0.5 1 1.5 2 2.5collo Ca / routine Ca, Cu XRF

collo

Ca

/ ro

utin

e C

a,

Mo

XR

F

2004

2005

Phoenix

0.5

0.75

1

1.25

1.5

0 0.5 1 1.5 2collo Ca / routine Ca, Mo XRF

collo

PM

2.5

/ rou

tine

PM

2.5,

gr

av

2004

2005

Phoenix

Minimum Detectable Limits Minimum Detectable Limits (mdl)(mdl)

Silicon

0.0010

0.0100

0.1000

1.0000

Average Concentration (ug m-3)

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

Re

lativ

e D

iffe

r en

ce

TRCRSAFOPMRFPHOEOLYMMEVEMDL

SITES

Chlorine

0.0001 0.0010 0.0100 0.1000Average Concentration (ug m-3)

-1.5

-1.0

-0.5

0.0

0.5

1.0

1.5

Re

lativ

e D

if fe

r en

ce

TRCRSAFOPMRFPHOEOLYMMEVEMDL

SITES

SpeciesCount > 3*MDL

Estimated Uncertainty

Collocated Precision

Collocated/ Estimated

Si 533/577 10.9% 40.5% 3.7Cl 105/577 12.1% 68.4% 5.7

Entire network January - April

0

0.5

1

0.1 1 10 100[Si val x (Al/Si)crust] / Al mdl

frac

tion

dete

cted

2005

2004

2003

  Original Criteria Both Detected

SpeciesNo. Pairs

Collo / Estimated

No. Pairs

Collo / Estimated

Na 15 2.0 71 1.1

Mg 9 - 45 1.0

Al 257 5.0 212 1.7

Si 533 3.7 500 2.0

P 21 7.3 3 -

S 577 1.6 577 1.6

Cl 105 5.6 95 1.4

K 577 2.3 577 2.3

Ca 577 3.0 576 2.9

Ti 528 2.6 520 1.8

V 457 1.2 485 1.0

Cr 108 2.0 195 1.1

Mn 529 2.0 546 1.8

Fe 577 3.3 577 3.3

  Original Criteria Both Detected

SpeciesNo. Pairs

Collo / Estimated

No. Pairs

Collo / Estimated

PM2.5 539 1.0 577 0.6

H 577 1.3 577 1.3

Ni 222 2.8 359 1.6

Cu 499 2.5 512 1.8

Zn 575 4.0 571 3.7

As 60 1.1 275 0.9

Se 406 1.1 500 1.1

Br 576 1.2 577 1.2

Rb 64 1.2 256 0.8

Sr 266 2.1 469 1.3

Zr 5 - 21 1.2

Pb 541 1.8 545 1.5

0.00001

0.0001

0.001

0.01

0.1

Jan-02 Jan-03 Jan-04 Jan-05

Cu

, u

g/m

3even oddBADL1

NON -DETECT

YNA YMG YAL YSI YP YS YCL YK YCA YTI YV YCR YMN YFE

percent of 1/04 samples throughout network yielding 'quantitative' values (ERR/VAL < 10%)0% 0% 13% 74% 1% 100% 10% 99% 98% 67% 15% 1% 49% 99%

percent of quantitative values below field blank (FB) levels95%ile 1/04 FB 2.9% 5.8% 0.7% 0.2% 7.4%90%ile 1/04 FB 5.1% 0.1% 3.6%75%ile 1/04 FB 3.7% 1.0%50%ile 1/04 FB 1.5%percent of quantitative values below acceptance test (AT) levels95%ile 10/04 AT 0.3% 0.7% 0.3% 2.0%90%ile 10/04 AT 0.2% 0.6%75%ile 10/04 AT

FE NI CU ZN AS PB SE BR RB SR Y ZR CD

percent of 1/04 samples throughout network yielding 'quantitative' values (ERR/VAL < 10%)99% 9% 34% 99% 3% 37% 26% 91% 0% 9% 0% 0% 0%

percent of quantitative values below field blank (FB) levels95%ile 1/04 FB 5.7% 35.4%90%ile 1/04 FB 0.9% 20.1%75%ile 1/04 FB 3.7%50%ile 1/04 FB 0.1%percent of quantitative values below acceptance test (AT) levels95%ile 10/04 AT 0.2% 26.0%90%ile 10/04 AT 3.2%75%ile 10/04 AT

Carbon Fraction

Analytical Uncertainty, uanalytic

2004 Replicate Precision

Precision/ Uncertainty

OC1 27% 41% 1.5

OC2 16% 17% 1.1

OC3 11% 19% 1.7

OC4 13% 20% 1.5

OP 27% 50% 1.9

EC1 13% 24% 1.9

EC2 26% 31% 1.2

EC3 40% 66% 1.7

OC - 11% -

EC - 21% -

Conclusions and Future Conclusions and Future WorkWork• Estimated uncertainties are not accurate Estimated uncertainties are not accurate

for most speciesfor most species– Additional sampling-related sources of Additional sampling-related sources of

uncertainty must be considereduncertainty must be considered– Sample deposit uniformity must be measuredSample deposit uniformity must be measured– Cyclone collection efficiency must be Cyclone collection efficiency must be

modeledmodeled

• Minimum detectable limits (mdl’s) are too Minimum detectable limits (mdl’s) are too low for several specieslow for several species– Collocated data will be used to evaluate mdl’sCollocated data will be used to evaluate mdl’s

Species Species with with Decent Decent AgreemenAgreement between t between Precision Precision and and UncertaintUncertaintyy

SpeciesEstimated

UncertaintyCollocated Precision

Collocated/ Estimated

PM2.5 7% 6% 1.0PM10 5% 8% 1.6Na 25% 50% 2.0S 5% 8% 1.6V 18% 21% 1.2Mn 12% 24% 2.0As 14% 15% 1.1Pb 12% 22% 1.8Se 14% 15% 1.1Br 7% 9% 1.2Rb 24% 28% 1.2H 6% 8% 1.3Nitrate 8% 10% 1.3Sulfate 5% 4% 0.9OC1 33% 44% 1.3OC2 22% 18% 0.8OC3 17% 24% 1.5OC4 17% 26% 1.5OP 29% 45% 1.5EC1 16% 24% 1.5EC2 31% 37% 1.2OC 12% 17% 1.5EC 12% 22% 1.8TC 11% 17% 1.5