applied problem solving and research using statistical...
TRANSCRIPT
![Page 1: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/1.jpg)
Applied Problem Solving andResearch Using StatisticalMethods with NIST Examples
Adam L. Pintar
September, 2016
![Page 2: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/2.jpg)
Introduction
![Page 3: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/3.jpg)
About Me
Grew up in Kansas
Education
Pittsburg State University: Mathematics
: Statistics
Family
3/71
![Page 4: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/4.jpg)
About NIST
NIST’s mission is to promote U.S. innovation and industrialcompetitiveness by advancing measurement science,standards, and technology in ways that enhance economicsecurity and improve our quality of life.
NIST is the national metrology institute (NMI) for the UnitedStates. As an NMI, NIST
Maintains primary measurement standards for the seven baseunits in the SI system of units and for derived unitsOffers calibration services and measurement standards tosupport international tradeDevelops new measurement technologies
The Institute for National Measurement Standards of theNational Research Council is the Canadian counterpart
4/71
![Page 5: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/5.jpg)
NIST Campus
5/71
![Page 6: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/6.jpg)
About SED
Churchill Eisenhart was the firstChief
Originally Statistical EngineeringLaboratory
1947
Lola Deming was a foundingmember too
6/71
![Page 7: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/7.jpg)
More History
W. J. Youden (Right)
J. Cameron (Left)
Graeco-Latin Square
Enamel on metal
7/71
![Page 8: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/8.jpg)
Outline
Experiment Design
X-ray CT for detecting additive manufacturing (3D printing)defectsInteractive discussion (choosing factors and levels)Accelerated degradation of polymeric materials
Exploratory Data Analysis
Accelerated degradation of polymeric materialsVolumetric versus RECIST length tumor measurementsStandard reference material heterogeneity
Probability/Stochastic Modeling
Standard reference material heterogeneityDistribution of peak pressureStandard reference material valuesInteractive discussion (choosing prior distributions)
Multidisciplinary projects
Localizing leaks of geologically sequestered CO2
8/71
![Page 9: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/9.jpg)
Experiment Design
![Page 10: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/10.jpg)
X-ray CT for detecting additive manufacturing (3Dprinting) defects
![Page 11: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/11.jpg)
Goal
Best practices for measuring void size
11/71
![Page 12: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/12.jpg)
Question 1
What factors influence the measurements?
12/71
![Page 13: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/13.jpg)
List of Factors – Version 1
CT acquisition
1. Voltage2. Current3. Filter type4. Filter thickness5. Magnification6. Frame rate7. Number of images per projection8. Detector type9. Pixel size
10. Scintillator type11. Scintillator thickness
Reconstruction
1. Algorithm2. Center of rotation3. Beam hardening correction4. Scattering correction
Artifact
1. Material2. Flaw size3. Flaw shape
Image processing
1. Smoothing filter2. Thresholding algorithm
20 factors!
13/71
![Page 14: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/14.jpg)
Available Resources
Able to produce about 20 images
Need 220 ≈ 1m runs for a 2-level full factorial experiment
The closest we can get to 20 runs is a 220−15 fractionalfactorial
32 runsMain effects confounded with 2 factor interactionsNo estimate of pure error
14/71
![Page 15: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/15.jpg)
List of Factors – Version 2
CT acquisition
1. Voltage (numeric)2. Current (numeric)3. Magnification (numeric)4. Frame rate (numeric)5. Number of images per projection (numeric)
Reconstruction
1. Algorithm (categorical)
15/71
![Page 16: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/16.jpg)
Design
26−2 fractional factorial with 4“center” runs
Main effects not confoundedwith 2 factor interactions2 df pure error estimateSome information about 2factor interactions
## volt curr mag fr n_img alg
## 1 0 0 0 0 0 -1
## 2 0 0 0 0 0 1
## 3 -1 -1 1 1 1 1
## 4 1 1 -1 1 -1 1
## 5 1 -1 1 -1 -1 1
## 6 -1 -1 1 -1 1 -1
## 7 1 1 1 1 1 1
## 8 -1 1 -1 -1 1 1
## 9 1 -1 1 1 -1 -1
## 10 -1 -1 -1 -1 -1 -1
## 11 0 0 0 0 0 -1
## 12 1 1 1 -1 1 -1
## 13 1 -1 -1 -1 1 1
## 14 -1 1 1 1 -1 -1
## 15 -1 1 -1 1 1 -1
## 16 1 1 -1 -1 -1 -1
## 17 1 -1 -1 1 1 -1
## 18 -1 1 1 -1 -1 1
## 19 -1 -1 -1 1 -1 1
## 20 0 0 0 0 0 1
16/71
![Page 17: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/17.jpg)
Computation
R is a language and environment for statistical computing andgraphics. . . .
17/71
![Page 18: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/18.jpg)
2-level Fractional Factorial Designs in R
FrF2 function from the R package FrF2
factor_names <- c('volt', 'curr',
'mag', 'fr',
'n_img', 'alg')
my_design <- FrF2(factor.names = factor_names,
resolution = 4, ncenter = 4)
18/71
![Page 19: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/19.jpg)
## volt curr mag fr n_img alg
## 1 0 0 0 0 0 0
## 2 0 0 0 0 0 0
## 3 -1 -1 1 -1 1 -1
## 4 1 1 -1 1 -1 1
## 5 -1 -1 -1 1 -1 1
## 6 1 -1 -1 1 1 -1
## 7 1 -1 1 -1 -1 1
## 8 1 1 1 -1 1 -1
## 9 -1 -1 -1 -1 -1 -1
## 10 1 -1 1 1 -1 -1
## 11 0 0 0 0 0 0
## 12 -1 1 -1 -1 1 1
## 13 -1 1 1 -1 -1 1
## 14 1 1 1 1 1 1
## 15 1 -1 -1 -1 1 1
## 16 -1 1 -1 1 1 -1
## 17 -1 1 1 1 -1 -1
## 18 1 1 -1 -1 -1 -1
## 19 -1 -1 1 1 1 1
## 20 0 0 0 0 0 0
## class=design, type= FrF2.center
19/71
![Page 20: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/20.jpg)
design.info(my_design)$alias
## $legend
## [1] "A=volt" "B=curr" "C=mag" "D=fr" "E=n_img" "F=alg"
##
## $main
## character(0)
##
## $fi2
## [1] "AB=CE=DF" "AC=BE" "AD=BF" "AE=BC" "AF=BD" "CD=EF"
## [7] "CF=DE"
20/71
![Page 21: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/21.jpg)
Interactive DiscussionChoosing Factor Levels
![Page 22: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/22.jpg)
Accelerated Degradation of Polymeric Materials
![Page 23: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/23.jpg)
Goal
Lab measurements
Field measurements
23/71
![Page 24: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/24.jpg)
Two Experiments
Laboratory
Precisely control/measure a few factors
Field
Environmental variationMimic building movement
Focus on Laboratory
24/71
![Page 25: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/25.jpg)
Lab Factors
Light intensity
Temperature
Humidity
Mechanical strain
25/71
![Page 26: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/26.jpg)
Design
Light intensity – Fixed at max
Temperature – 4 levels
Humidity – Fixed
Mechanical strain – 4 levels
42 full factorial
26/71
![Page 27: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/27.jpg)
0 5 10 15 20
2025
3035
4045
50
Mechanical Strain (%)
Tem
pera
ture
(C
)
27/71
![Page 28: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/28.jpg)
Exploratory Data Analysis
![Page 29: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/29.jpg)
Accelerated Degradation of Polymeric Materials
![Page 30: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/30.jpg)
Raw Data
Temp: 21Strain: 0
0.0
0.4
0.8
1.2
0 20 40 60 80
Chamber 1Chamber 2Chamber 3Chamber 4
Temp: 21Strain: 5
Temp: 21Strain: 11
0 20 40 60 80
Temp: 21Strain: 21
Temp: 31Strain: 0
Temp: 31Strain: 5
Temp: 31Strain: 11
Temp: 31Strain: 21
0.0
0.4
0.8
1.2
Temp: 41Strain: 0
0.0
0.4
0.8
1.2
Temp: 41Strain: 5
Temp: 41Strain: 11
Temp: 41Strain: 21
Temp: 51Strain: 0
Temp: 51Strain: 5
0 20 40 60 80
Temp: 51Strain: 11
Temp: 51Strain: 21
0.0
0.4
0.8
1.2
0 20 40 60 80
Exposure Days
Mod
ulus
Rat
io
30/71
![Page 31: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/31.jpg)
Volumetric versus RECIST Length TumorMeasurements
![Page 32: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/32.jpg)
Experiment
32/71
![Page 33: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/33.jpg)
Raw Data
Volume (cm3)
Mas
s (g
)
2 3 4 5 6
23
45
6
RECIST (mm)
20 25 30 35
23
45
6
Color differentiates diapers
Expect straight lines with positive slopes
33/71
![Page 34: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/34.jpg)
Standard Reference Material Heterogeneity
![Page 35: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/35.jpg)
SRMs
35/71
![Page 36: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/36.jpg)
SRM Certificate
36/71
![Page 37: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/37.jpg)
Coal Bottle-to-Bottle Differences (Heterogeneity)
2 4 6 8 10
3.2
3.4
3.6
3.8
4.0
4.2
4.4 Bromine
Bottle
mg/
kg
ObservationsGrand mean
37/71
![Page 38: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/38.jpg)
Probabilistic/StochasticModelling
![Page 39: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/39.jpg)
Standard Reference Material Heterogeneity
![Page 40: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/40.jpg)
Bottle-to-Bottle Differences (Heterogeneity)
2 4 6 8 10
3.0
3.5
4.0
4.5
Bromine
Bottle
mg/
kg
ObservationsGrand meanGrand mean CIBottle mean CI
ANOVA p−value: 0.074
40/71
![Page 41: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/41.jpg)
p-value
H0: Bottles all the sameHA: At least one bottle different
anova(lm(raw_data$mg_kg ~ factor(raw_data$bottle)))
## Analysis of Variance Table
##
## Response: raw_data$mg_kg
## Df Sum Sq Mean Sq F value Pr(>F)
## factor(raw_data$bottle) 9 1.62004 0.180005 2.6253 0.07438 .
## Residuals 10 0.68565 0.068565
## ---
## Signif. codes: 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1
41/71
![Page 42: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/42.jpg)
Power
Probability of concluding at least one bottle is different whenthat is the true state of nature
power.anova.test(groups = 10, n = 2,
between.var = c(0.5, 1, 1, 2), within.var = 1,
sig.level = c(0.01, 0.05, 0.15, 0.1) )
##
## Balanced one-way analysis of variance power calculation
##
## groups = 10
## n = 2
## between.var = 0.5, 1.0, 1.0, 2.0
## within.var = 1
## sig.level = 0.01, 0.05, 0.15, 0.10
## power = 0.08086632, 0.51463217, 0.77715565, 0.93728416
##
## NOTE: n is number in each group
42/71
![Page 43: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/43.jpg)
Peak Pressure
![Page 44: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/44.jpg)
Structural Design
44/71
![Page 45: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/45.jpg)
Experimental Data
0 20 40 60 80 100
−3.
0−
2.0
−1.
00.
0
Time (s)
Pre
ssur
e
45/71
![Page 46: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/46.jpg)
Data Processing
20 40 60 80 100
0.0
0.5
1.0
1.5
2.0
2.5
3.0
Time (s)
Pre
ssur
e
46/71
![Page 47: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/47.jpg)
Simulated Data
0 50 100 150 200
0.0
1.0
2.0
3.0
Pre
ssur
eOriginal Data Set
0 50 100 150 200
0.0
1.0
2.0
3.0
Pre
ssur
e
Fake Data Set #1
0 50 100 150 200
0.0
1.0
2.0
3.0
Time (s)
Pre
ssur
e
Fake Data Set #2
47/71
![Page 48: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/48.jpg)
Distribution of the Peak
Distribution of the Peak Value
Peak Value
Den
sity
3.0 3.5 4.0 4.5
0.0
0.5
1.0
1.5
Mean
48/71
![Page 49: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/49.jpg)
Uncertainty in the Distribution of the Peak
Distribution of the Peak Value
Peak Value
Den
sity
3.0 3.5 4.0 4.5
0.0
0.5
1.0
1.5
Mean
Bootstrap Replicates80% CI for the Mean
49/71
![Page 51: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/51.jpg)
Standard Reference Material Values
![Page 52: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/52.jpg)
Bayesian methods – high level
Two sources of information
DataSubject matter expertise (Prior)
Bayes rule tells us how to combine them
52/71
![Page 53: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/53.jpg)
Hard Rock Mine Waste
Governor Basin, Colorado53/71
![Page 54: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/54.jpg)
Ag (Silver) Raw Data
7075
80m
g/kg
ICPMS Lab 1 ICPOES ICPMS Lab 2 INAA
54/71
![Page 55: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/55.jpg)
Posterior With Flat Prior
40 60 80 100 120
0.00
0.02
0.04
0.06
0.08
Ag (Silver)
mg/kg
Den
sity
95% interval [55, 104]55/71
![Page 56: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/56.jpg)
The Problem
7075
80m
g/kg
ICPMS Lab 1 ICPOES ICPMS Lab 2 INAA
56/71
![Page 57: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/57.jpg)
Prior Information
0.0 0.2 0.4 0.6 0.8 1.0
01
23
45
Logarithmic Units
Den
sity
InfomativeFlat
Quality control data (Log scale)57/71
![Page 58: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/58.jpg)
Posterior With Informative Prior
60 70 80 90 100 110
0.00
0.02
0.04
0.06
0.08
0.10
Ag (Sliver)
mg/kg
Den
sity
95% interval [66, 83]58/71
![Page 59: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/59.jpg)
Interactive DiscussionChoosing Prior Distributions
![Page 60: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/60.jpg)
Multidisciplinary Projects
![Page 61: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/61.jpg)
Goal
Detect and locate leaks of CO2 stored in geological formations
61/71
![Page 62: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/62.jpg)
Test Site in Montana
The data that were analyzed come from a similar site in Ft.Wayne Indiana using the same equipment
62/71
![Page 63: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/63.jpg)
Likelihood at Ft. Wayne
Grid is clearly visibleExpected no signal, but found a strong one
63/71
![Page 64: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/64.jpg)
Collaborators
Zachary H. Levine (NIST)
Jeremy T. Dobler (Harris Corp.)
Nathan Blume (Harris Corp.)
Michael Braun (Harris Corp.)
T. Scott Zaccheo (Atmospheric and Environmental Research)
Timothy G. Pernini (Atmospheric and EnvironmentalResearch)
64/71
![Page 65: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/65.jpg)
Reference
Levine, Z. H., Pintar, A. L., Dobler, J. T., Blume, N., Braun, M.,Zaccheo, T. S., Pernini, T. G., ”The Detection of Carbon DioxideLeaks Using Quasi-tomographic Laser Absorption SpectroscopyMeasurements in Variable Wind,” *Atmospheric MeasurementTechniques*, **9**, 1627–1636, 2016, http://www.atmos-meas-tech.net/9/1627/2016/amt-9-1627-2016.pdf.
65/71
![Page 66: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/66.jpg)
Summary
Three main toolboxes
Experiment designExploratory data analysisStochastic/Probabilistic modelling
Toolboxes interact with each other
Multidisciplinary teams
66/71
![Page 67: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/67.jpg)
Questions
67/71
![Page 68: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/68.jpg)
Question One
Thank you Michelle T.
Synopsis
Purchase four physical standards for calibrating liquidchromatography instrumentsCombine the standards into a single new “secondary” standardHow to verify that the combination procedure does not changethe concentrations listed on the certificates
68/71
![Page 69: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/69.jpg)
Simple Graphical Approach
Quality control checks when measuring candidate SRMs
810
1214
16
µg/k
g
Certificate Measurements
69/71
![Page 70: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/70.jpg)
Formalization of the Simple Approach
H0: No differenceHA: Some difference
Review paper
Rukhin, A. L., “Assessing Compatibility of Two Laboratories:Formulations as a Statistical Hypothesis Testing Problem,”Metrologia, 50, 49–59 (2013).
Potential problem
Can find strong evidence of a difference but not strongevidence of equivalence
70/71
![Page 71: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/71.jpg)
Second FormalizationH0: Difference is practically importantHA: Difference is not practically important
ReferenceAnderson-Cook, C. M. and Borror, C. M., “The DifferenceBetween “equivalent” and “not different,”” QualityEngineering, 28, 249–262 (2016).
Some modification necessary8
1012
1416
µg/k
g
Certificate Measurements
71/71
![Page 72: Applied Problem Solving and Research Using Statistical ...asq.org/statistics/2016/12/applied-problem-solving-and-research-using... · Research Using Statistical Methods with NIST](https://reader030.vdocuments.site/reader030/viewer/2022040908/5e7f8b6269211d3b0119ea3b/html5/thumbnails/72.jpg)