quality assurance/ quality control nate herbst southern ute indian tribe
TRANSCRIPT
Quality Assurance/ Quality Control
Nate Herbst
Southern Ute Indian Tribe
2
Intro to QA/QC
• Getting good data requires many different steps– Data quality objectives (DQOs) developed
• DQOs for ozone being developed• Measurement quality objectives (MQOs) for ozone exist
– Analysis begun (after correct calibration)– QC checks performed– QA conducted
3
Data Quality Objectives (7 steps)
• State problem– Define why monitoring is needed– Create team and purpose
• Identify decision– What decision will be made with data?
• Identify decision inputs– What data necessary to make decision?
• Define boundaries– What are study area boundaries?
4
Data Quality Objectives (cont.)
• Develop decision rule– What conditions will require action (action level)?
• Specify decision error limits– What margin of error is allowable
• Optimize monitoring design– Develop most cost-effective method of reaching
DQOs
• EPA hasn’t yet defined DQOs for ozone analysis
5
DQOs (cont.)Diagram of DQO steps(Diagram courtesy of U.S. Department of Energy – DQO homepage)
6
Measurement Quality Objectives (MQOs)• EPA has ozone analysis MQOs• Use these in element 7 of your QAPP• MQOs in a nutshell
– Shelter temperature kept between 20-30oC ± 2oC– Analyzer must be reference or equivalent method– Lower detectable limit 0.01ppm, noise 0.005ppm– Data completeness 75% of hourly values between
9:01am and 9:00pm (for the ozone season)– Transfer standard certification ±4% or 4ppb
(whichever is greater)
Thanks to Melinda Ronca-Batista (ITEP)
7
MQOs (cont.)
– Transfer standard re-certification to primary std.dev 1.5%
– Local primary standard certification ±5% of reference
– EPA reference photometer regression slope 1.00 ± 0.01
– Zero air free of O3 and anything that might react with O3
8
MQOs (cont.)
– Ozone analyzer calibration • Z/S check zero ±10ppb, span ± 15%• 5pt calibration linearity error ±5%
– Performance (NPAP) mean absolute difference ± 15%
– Precision (quarterly) 95% CI < ±15%– Audits (annually) 95% CI < ±20%
9
Quality Assurance Project Plans (QAPPs)
• Contain 24 “elements” – Element 7 is where MQOs go– Cut and paste from red book
• Ensure data quality• Required by EPA• Developed by program approved by EPA• They must be followed!
– No good if not followed
10
Documentation
• Document everything!!!• Documentation in
– Logbooks– Site folders– QA/QC field forms– Anywhere else you think is appropriate
• QA/QC – document standard values and response
11
Documentation (cont.)
• Document repairs, checks, fine tunes• Document site conditions• Document everything that could ever be
important• Write only in pen (black if possible)• Cross out errors with a single line
12
Linearity
y = x
y = x
R2 = 1
0
1
2
3
4
5
6
0 2 4 6X axis
Y axismb
• Slope = rise over run
• m = slope• b = intercept (where
the trend-line crosses the Y axis)
• r2 close to 1 shows correlation
y = mx+b
y = 2.0829x + 1.9095R2 = 0.997
02468
101214
0 2 4 6X axis
Y axis
13
Instrument Calibration
• Measurements require point of reference• Measurement without standard is impossible• Calibration involves setting instrument to
known level• Calibrations performed fairly regularly
– When monitoring is begun– When repairs or maintenance are performed– When precision checks or audits show need
• Calibrations must be done correctly
14
Calibrations (cont.)
• Calibration = setting analyzer to standard– Data only good within linear range (~0-0.400ppm)
• Calibration followed by a 5-pt check• Analyzer must agree with standard at all 5 pts
– Linearity error < 5%– See next slide on linearity
• Monitoring begins after calibration
Note: Never initiate monitoring without calibration
15
Pre-Calibration Check5pt check
y = 1.1368x + 0.0203R2 = 0.98
0.000
0.100
0.200
0.300
0.400
0.500
0.000 0.100 0.200 0.300 0.400 0.500Standard
Analyzer
• Not always necessary
• Can do 5-pt check • Analyzer must be
calibrated• The r2 value and %
differences for each point are unacceptable
Standard Analyzer % dif.
0.000 0.000 0.0
0.080 0.100 -25.0
0.150 0.220 -46.7
0.250 0.340 -36.0
0.350 0.400 -14.3
0.400 0.460 -15.0
16
Instrument Calibration
• Calibrate instrument to the standard• Use calibration point near URL
– Setting low produces large error at URL
• Set standard to ~0.400 ppm • Let analyzer stabilize• Calibrate analyzer• Do new 5-pt check
17
Post-Calibration 5-pt Check
• Is analyzer response within 7% at each point?
• Would you put this analyzer online?
Calibration
y = 1.0145x - 0.0006R2 = 0.9997
0.000
0.200
0.400
0.600
0.000 0.100 0.200 0.300 0.400 0.500Standard
Analyzer
Standard Analyzer % dif.
0.000 0.000 0.0
0.080 0.081 -1.3
0.150 0.153 -2.0
0.250 0.248 0.8
0.350 0.354 -1.1
0.400 0.408 -2.0
18
Quality Control (QC)
• QC involves “in-house” verifications• Also referred to as precision checks• Verifications are comparisons between
transfer standard and analyzer– Relative % difference within allowable margin?
• Verifications determine monitoring repeatability– Standard deviation
• Different types of verifications
19
QC (cont.)
• Level 1: 40 CFR, Pt. 58, App. A, Table A-1 defines ozone verification requirements (for SLAMS)– Biweekly response check between 0.08 and 0.1
ppm • Comparison between analyzer and standard
– Determines repeatability
• Level 2: “extra” precision checks– Weekly “span level” (~80% URL) checks– Quarterly 5-pt checks– Determines analyzer performance trends
20
Quality Assurance (QA)
• QA involves “external” checks• Referred to as “audits”• Audits involve comparison between transfer
standard and analyzer– Accuracy levels must be within ±10%
• Audits determine how close monitoring gets to actual values
• Different types of audits
21
QA (cont.)• 40 CFR, Pt. 58, App. A, Table A-1 defines
ozone audit requirements (for SLAMS)– Annual (and other) response checks at multiple
points• 0.03-0.08 ppm• 0.15-0.2 ppm• 0.35-0.45 ppm
– Comparison between analyzer and external standard
– Audits should include zero check
22
QA (cont.)
• Different types of audits– By reporting organization (RO) certified by
RO– By RO certified by other than RO– By other than RO certified by other than
RO
23
Precision & Accuracy (P&A) Data
• Precision data come from biweekly precision checks
• Accuracy data come from annual and other audits
• P&A data validate ambient data• P&A data must be included in AQS data
submittals
24
Siting Criteria
• Data quality depends on correct siting of all instrumentation
• Specific instrument siting guidelines• Following guidelines is vital part of quality
assurance and control• We’ll learn more about these guidelines in the
next presentation
25
Summary • Establish DQOs• Develop QAPP
– Get it approved by EPA
• Follow your QAPP• Conduct bi-weekly precision checks
– Conduct level 2 checks to follow monitor trends
• Participate in annual audits and others if possible
• Data quality will be guaranteed