robert l. sielken jr., ph.d. sielken & associates consulting inc
DESCRIPTION
Experiences Helping Develop More Effective Regulations via Interactions Between the Public, Universities, Regulated Entities and Regulators. Robert L. Sielken Jr., Ph.D. Sielken & Associates Consulting Inc 3833 Texas Avenue, Suite, 230, Bryan, TX 77802 - PowerPoint PPT PresentationTRANSCRIPT
Sielken & Associates Consulting, Inc.1
Robert L. Sielken Jr., Ph.D.Sielken & Associates Consulting Inc
3833 Texas Avenue, Suite, 230, Bryan, TX 77802Tel: 979-846-5175; Fax: 979-846-2671; Email:
Air Toxics Workshop II: Air Toxics ResearchImplications of Research on Policies to Protect Public Health
Session II: Interactive Processes in Toxicity AssessmentsHouston, Texas
Tuesday 10:20 am - 12:00 Noon, June 12, 2007
Experiences Helping Develop More Effective Regulations via Interactions
Between the Public, Universities, Regulated Entities and Regulators
Sielken & Associates Consulting, Inc.2
Interactive Processes
Scientists
Industrial hygienists and scientists
Academic researchers
Concerned citizens
Regulators and Risk Managers
Consultants, specialists
Sielken & Associates Consulting, Inc.3
As a statistician, researcher, consultant, and university professorin the field of quantitative human and environmental risk assessment,I have had the opportunity to interact with risk assessors and managers in numerous contexts:
States:
TexasFloridaCaliforniaMinnesotaWisconsinMichiganIllinoisIndianaOhioPennsylvaniaNew Yorketc.
Federal Government:
CongressEPAFDAOSHANIHNIEHSNCTRNAS/NRCetc.
Universities:
Texas A&MU. of TexasHarvard Center for Risk Analysis
etc.
Task Forces:
Cancer Risk AssessmentBenchmark DoseFood ProtectionGreat LakesILSIACCCMAAIHCSOTSRASETACISRTPASAetc.
Litigation:
MissouriTexasLouisianaVirginiaCaliforniaColoradoArizonaMississippiHawaiiDelawareCanadaetc.
Industry:> 100 Clients
Sielken & Associates Consulting, Inc.4
My Job:
Bridger between Risk Assessors and Managersand Other Scientists
-- Ask Questions No One Else Dares Ask
Bridger between Regulators and those being Regulated
-- Be Someone All Sides Respect and Trust
Sielken & Associates Consulting, Inc.5
My Job:
Help all those involved
to avoid the feeling
that they are being “hurt” or “fooled”
by someone’s
incorrect, inappropriate, incomplete, or inadequate
treatment of the available data.
Sielken & Associates Consulting, Inc.6
My Job:
Help Risk Assessors and Risk Managers
-- recognize the limitations of default methodology
-- understand the opportunities available due torecent advances in risk assessment methodologyand risk management techniques
-- avoid the pitfalls associated with poorly understoodmathematical and statistical procedures
Sielken & Associates Consulting, Inc.7
Partial List of Findings that Impacted the Most RecentUS EPA Draft Risk Assessment for Ethylene Oxide:
Common Failures:
Failure to use the newest data.
Failure to use all of the data.
Failure to use a valid dose-response modeling approach.
Errors in calculations.
Results fail simple reality checks.
Sielken & Associates Consulting, Inc.8
The most common implementation of the age-dependentadjustment factor (ADAF) is mathematically incorrect.
Excess risk calculations for incidence using estimated dose-response models for mortality are inappropriate.
The BEIR IV life-table methodology for calculating excess risk ismathematically correct when the response of concern is mortality but is incorrect when the response of concern is incidence.
Conclusions about lower-dose risks based onhigh-to-low-dose extrapolation using fitted dose-response modelsdominated by the high-dose portion of the datamay be contradicted by the observed lower-dose data.
It is critically important to do dose-response assessment using theexposure and outcome data for the individuals in the cohortrather than on summaries of groups of individuals (e.g., odds ratios).
Partial List of Findings that Impacted the Most RecentUS EPA Draft Risk Assessment for Ethylene Oxide:
Sielken & Associates Consulting, Inc.9
Assuming an 85-year exposure lifetime instead of a 70-year exposure lifetime substantially impacts calculations ofexcess risk for many toxic endpoints (e.g., most cancers).
Assumptions frequently dominate excess risk calculations.
In order to fairly compare the risks of two substances,the risks must both be calculated using the same assumptions.
Partial List of Findings that Impacted the Most RecentUS EPA Draft Risk Assessment for Ethylene Oxide:
Sielken & Associates Consulting, Inc.10
Partial List of Findings that Impacted the Most RecentUS EPA Draft Risk Assessment for Ethylene Oxide:
The existence of repair and other background defense mechanismscan imply that the extrapolation below a point of departure (POD)should not be done linearly.
Sielken & Associates Consulting, Inc.11
Partial List of Findings that Impacted the Most RecentUS EPA Draft Risk Assessment for Ethylene Oxide:
The existence of repair and other background defense mechanismscan imply that the extrapolation below a point of departure (POD)should not be done linearly.
Sielken & Associates Consulting, Inc.12
Partial List of Findings that Impacted the Most RecentUS EPA Draft Risk Assessment for Ethylene Oxide:
You cannot conclude what the SHAPE of the dose-response relationship is
if you only fit models of a specified shape to the data.
For example,if you only fit linear models, then the fitted shape is linear; however, that does not mean that the true shape of the dose-response relationship is linear.
Similarly, if you only fit supra-linear models, then the fitted shape is supra-linear; however, that does not mean that the true shape of the dose-response relationship is supra-linear.
Sielken & Associates Consulting, Inc.13
Partial List of Additional Findings that Impacted Risk Assessments:
Other Risk Assessments
Sielken & Associates Consulting, Inc.14
Risk extrapolations from occupational to environmental scenarios
need to account for the differences in these scenarios
especially with respect to
exposure magnitude, duration, and temporal spacing
as well as confounding factors like
exposures to other substances
and the number of high intensity tasks.
Sielken & Associates Consulting, Inc.15
Sielken & Associates Consulting, Inc.16
The default procedure used by some risk assessors(e.g., in EPA and California) to bound cancer potencies
is dominated by default assumptions and does not reflect the observed experimental data:
The linearized multistage model upper bound on the cancer slope (q1*)fails to adequately reflect the shape of the observeddose-response relationship and especially the outcomesin the low-dose region, which is the region ofprimary interest in risk assessment.
Sielken & Associates Consulting, Inc.17
20%
0% Dose0 0.25 0.50 1.0
Re
spo
nse
Fre
qu
en
cyLinearizedMultistage
Model Slope = 0.035
20%
0% Dose0 0.25 0.50 1.0
Re
spo
nse
Fre
qu
en
cy
LinearizedMultistage
Model Slope = 0.332
20%
0% Dose0 0.25 0.50 1.0
Re
spo
nse
Fre
qu
en
cy
LinearizedMultistage
Model Slope = 0.311
20%
0% Dose0 0.25 0.50 1.0
Re
spo
nse
Fre
qu
en
cy
LinearizedMultistage
Model Slope = 0.065
20%
0% Dose0 0.25 0.50 1.0
Re
spo
nse
Fre
qu
en
cy
LinearizedMultistage
Model Slope = 0.084
20%
0% Dose0 0.25 0.50 1.0
Re
spo
nse
Fre
qu
en
cy
LinearizedMultistage
Model Slope = 0.120
100% at High Dose
100% at High Dose
100% at High Dose
= observedresponse frequency
Although the outcomes for each of the 6 experiments are very different,the slopes q1* differ by less than 10 fold (one order of magnitude).
Sielken & Associates Consulting, Inc.18
This example and similar other exampleshave promoted several regulatory agenciesto emphasize
Best Estimates instead of Bounds
especially when the dose-response data are human data.
For example, emphasizing
the fitted dose-response modelinstead of an upper bound on that model
ECs instead of LECs
BMDs instead of BMDLs
Sielken & Associates Consulting, Inc.19
Sielken & Associates Consulting, Inc.20
Errors:
Almost any data collection involves some error
-- Measurement Error-- Reporting Error
etc.
Sielken & Associates Consulting, Inc.21
Errors usually cause
upper sample percentiles
to have an
OVERESTIMATION BIAS.
Sielken & Associates Consulting, Inc.22
Impact of Errors: Simple Example
True Concentration = 10
Concentration
Concentration with Error
Upper PercentileGreater ThanTrue Concentration
10
10
Sielken & Associates Consulting, Inc.23
The Impact of Errors on the Sample Percentiles is Least for Central Tendency and Greatest for Extreme Percentiles
1,000 Monte Carlo Samples of Size 1000F
req
ue
nc
y
20%
40%
0%
60%
80%
100%
99.9th Percentile
Fre
qu
en
cy
20%
40%
0%
60%
80%
100%
95th Percentile
Fre
qu
en
cy
20%
40%
0%
60%
80%
100%
90th Percentile
Fre
qu
en
cy
20%
40%
0%
60%
80%
100%
50th Percentile
Ratios: (Sample Percentile / True Percentile)
0 0.0010.01
0.10.9 1 1.1
10 1001000
Sielken & Associates Consulting, Inc.24
If the data collection involves errors, then the inflationary errors at the extremes of the data distributions make
the extreme percentiles of the assessment the shakiest foundation for good decision making
and the least reliable basis for differentiation between different chemicals or situations.
Sielken & Associates Consulting, Inc.25
Sielken & Associates Consulting, Inc.26
Lessons learned in the development of the
Integrated Endangerment Assessment /Risk Characterization
for the Rocky Mountain Arsenal (RMA)near Denver, Colorado
Sielken & Associates Consulting, Inc.27
Bounds on exposure should NOT be determined by simply evaluating an exposure equation or model with each exposure
parameter's distribution
replaced by
a bounding constant.
Sielken & Associates Consulting, Inc.28
For example,if visitation hours per lifetime were evaluated as
(Hours per Day)x (Days per Year)
x (Years per Lifetime)
then the 95th percentile of the corresponding probability distribution for Recreational Visitors to RMA would be approximately
200 hours per lifetime
Sielken & Associates Consulting, Inc.29
However, if each component variable was simply replaced by its 95th percentile, then
(Hours per Day)0.95
x (Days per Year)0.95
x (Years per Lifetime) 0.95
= 1200 hours per lifetime
or approximately 6 times greater than the true 95th percentile
200 hours per lifetime
Sielken & Associates Consulting, Inc.30
Furthermore, if each component variable was replaced by a default "reasonable maximum exposure" (RME) value, then
(Hours per Day)RME
x (Days per Year)RME
x (Years per Lifetime)RME
= 10,400 hours per lifetime
or more than 50 times greater than the true 95th percentile
200 hours per lifetime.
Sielken & Associates Consulting, Inc.31
Sielken & Associates Consulting, Inc.32
Exposure and risk characterizationscan be very differentdepending on whether or not variability is incorporated.
Since variability is a part of reality,the most realistic exposure and risk characterizationsincorporate variability.
Sielken & Associates Consulting, Inc.33
Distribution of Lifetime Average Daily DoseModeling With and Without Year-to-Year Variability
mg / kg / day
Fre
qu
enc
y
10%
20%
30%
40%
0.000001
0.000005
0.00001
0.00005
0.0001
0.0005
0.001
WithYear-to-Year Variability
WithoutYear-to-Year Variability
Sielken & Associates Consulting, Inc.34
Sielken & Associates Consulting, Inc.35
Potentially exposed populations are comprised of people who do NOTconduct their lives in an identical fashion, but people who vary widely in theiractivities, diets, hobbies, desires, preferences,interests, obligations, and motivations. Such variation is more fully reflected in a distribution than a constant.
Sielken & Associates Consulting, Inc.36
Examples of Quantitative Impact of Incorporating the Exposure Variability
Within the Population
Sielken & Associates Consulting, Inc.37
The quantitative impact on the distribution of the lifetime average daily dose from drinking water ingestionof assuming that the exposure duration is either 70 years or less than a full lifetime (e.g., one residence duration)
0
20
40
60
80
Per
cen
tag
e
Duration ofExposure
70 years
One Residence Duration
0 to 1E-10
1E-91E-8
1E-70.000001
0.00001
0.0001
0.0010.01
Lifetime Average Daily Dose (mg / kg / day)
Sielken & Associates Consulting, Inc.38
Pesticide: Water Concentration (ppb)
The distributional characterization of the concentration in the drinking water in 9 of the 18 major use states with sample data in the data base:Variability from State to State and Person to Person within a State
MD
ILIN
HI
CADE
FL
IA
KS
1E-10
1E-91E-8
1E-70.000001
0.00001
0.0001
0.001
0.010.1
1 10 100
Per
cen
tag
e
0
20
40
60
80
100
Sielken & Associates Consulting, Inc.39
NY
NE
MNMI
MONC
OH
PAWI
Pro
po
rtio
n
1.00
0.80
0.60
0.40
0.20
0.00
Margin of Exposure
10,000,000,000
1,000,000,000
100,000,000
10,000,000
1,000,000
100,000
10,000
1,000
100
101
The distribution of the concentration in the drinking water and theVariability from State to State and Person to Person within a Statecarries forward to the distribution of the margin of exposure associated with drinking water ingestion in 9 major use states
Sielken & Associates Consulting, Inc.40
Sielken & Associates Consulting, Inc.41
Hence, it can be important to incorporatetemporal, spatial and demographic variabilityinto exposure and risk characterizations.
Temporal, spatial and/or demographic variabilitycan have a major impact on exposure and risk distributions.
Sielken & Associates Consulting, Inc.42
Temporal Integration: Hypothetical Example: Target = Three Month Exposure
Drinking Water
Spring Summer Fall Winter Without TemporalIntegration
Food
+ + + + Combined
Non-Dietary
SpringAggregate
SummerAggregate
FallAggregate
CombinedAggregate
+ + + + Combined
Sielken & Associates Consulting, Inc.43
WithoutTemporalIntegration
WithTemporalIntegration
Temporal Integration: Hypothetical Example: Target = Three Month ExposureImportance of Pooling Exposures
in the Same Season versus Mis-Matched Seasons
Fre
qu
en
cy
10%
20%
30%
40%
0%
mg / kg / day
Sielken & Associates Consulting, Inc.44
Sielken & Associates Consulting, Inc.45
Populations are often comprised of subpopulations(e.g., males and females, andprivate well and community water supply users).
Population characterizations can incorporatethe size and other characteristics of the component subpopulations.
The distribution of exposures in the population is NOT THE SAME AS the distribution of exposures in the most exposed subpopulation..
Sielken & Associates Consulting, Inc.46
Suppose that a population
is comprised of
two subpopulations,
A and B
Sielken & Associates Consulting, Inc.47
0
10
20
30
40
50
0 5 10 15 20 25 30
Distributional Characterizations of a Population Comprised of Two Subpopulations:50% Subpopulation A, 50% Subpopulation B
Correct: A and B
Incorrect: A + BP
erce
nta
ge
AB
The distribution in the population comprised of subpopulations A and Bis not the distribution of the values for the sum of A and B.
Sielken & Associates Consulting, Inc.48
The relative sizes of
the subpopulations comprising
the population impact
the distributional characterization
of the population.
Sielken & Associates Consulting, Inc.49
Subpopulation A = 50% of PopulationSubpopulation B = 50% of Population
vsSubpopulation A = 90% of PopulationSubpopulation B = 10% of Population
vs Subpopulation A = 10% of PopulationSubpopulation B = 90% of Population
0 1 5 10 15 20 25 30
Per
cen
tag
e
50% A , 50% B
10% A, 90% B
90% A, 10% B
0
10
20
30
40
50
Sielken & Associates Consulting, Inc.50
The distribution of exposures in the population is not the same as the distribution of exposures in the most exposed subpopulation
PopulationDistribution
Most ExposedSubpopulationDistribution
0
10
20
30
40
50
Per
cen
tag
e
0 1 5 10 15 20 25 30
Sielken & Associates Consulting, Inc.51
CORN
Grower Commercial
M/LMixer / Loader
AApplicator
Aerial
M/L A M/L Pilot
Tree for Assessment of Pesticide Handler Exposure
Ground
OpenPour
ClosedSystem
OpenCab
ClosedCab
OpenPour
ClosedSystem
Both (M/L & A)
Open ClosedPour System
Open ClosedPour System
Open ClosedCab Cab
OpenCab
ClosedCab
OpenCab
ClosedCab
7/10 3/10 3/10 7/10 7/10 3/10
3/10 7/10 3/10 7/10 3/10 7/10 2/10 8/10 3/10 7/10
1/3 1/3 1/3
1/2 1/2 1/2 1/2
97% 3%
86% 14%
Sielken & Associates Consulting, Inc.52
PESTICIDE HANDLING Corn Production, Flowable Formulation
Population and Subpopulations
Margin of Exposure
ApplicatorMixer/Loader
Commercial Aerial Applicator
Mixer/Loader
Mixer/Loader & ApplicatorCommercial Ground
Mixer/LoaderApplicator
GrowersAll Pesticide Handlers
10,000,000,000
1,000,000,000
100,000,000
10,000,000
1,000,000
100,000
10,000
1,000
100
1010.00
0.20
0.40
0.60
0.80
1.00
Sielken & Associates Consulting, Inc.53
Sielken & Associates Consulting, Inc.54
Multiple Choice: Is 100
1) NEAR A + B2) GREATER THAN A + B3) LESS THAN A + B
4) NEAR the 95th Percentile of A + B5) GREATER THAN the 95th Percentile of A + B6) LESS THAN 95th Percentile of A + B
Answer: All of these outcomes are possible!
For example, suppose95th Percentile of A + 95th Percentile of B
= 50 + 50 = 100
When A and B are characteristics that have distributions, thenyou usually must use a technique like Monte Carlo simulationto determine the distribution of a combination of A and B(e.g., A + B). For example, you can’t determine the 95-th percentile of A+B simply by knowing the 95-th percentile of A and the 95-th percentile of B.
Sielken & Associates Consulting, Inc.55
Sielken & Associates Consulting, Inc.56
There is often more than one way to calculate a bound,and different ways can produce very different bounds.
-- not all bounds have the form: best estimate ± a few standard deviations
See your local statisticianfor the best bounding methodology!
Sielken & Associates Consulting, Inc.57
There can be a huge difference between anEstimated Risk
and a Bound on Risk.
Therefore, it can be very important to state whethera stated risk is an estimate or a bound.
Sielken & Associates Consulting, Inc.58
Statistical confidence limits are intended toreflect experimental variability
notmisspecified model families,
unsatisfied assumptions,alternative choices,
etc.
A statistical 95% upper confidence limit is not gospel.
-- On the one hand, it may capture only a small part of what is unknown.
-- On the other hand, it may be a gross exaggeration because of the method chosen for its calculation and the assumptions incorporated.
Sielken & Associates Consulting, Inc.59
Sielken & Associates Consulting, Inc.60
Some bounding procedures exaggerate risks far more for some underlying dose-response relationships than others.
For example, the linearized multistage model that is often usedto generate upper bounds (q1*) on the cancer potencygenerally exaggerates the risk
-- 1 to 2 fold for linear dose-response relationships
-- 5 to 10 fold for linear-quadratic dose-response relationships
-- 100 to 1,000 fold for quadratic dose-response relationships
Thus, bounds exaggerate the risks associated with “safer” substances more than they exaggerate the risks for “less safe” substances.
Thus, risk comparisons between substances should be based on best estimates rather than bounds.
Sielken & Associates Consulting, Inc.61
1,0000.5 1 2 5 100 20Ratio
50 100
100%Frequency
1,0000.5 1 2 5 100 20Ratio
50 100
100%Frequency
1,0000.5 1 2 5 100 20Ratio
50 100
100%Frequency
1%
94%
72%
6%
20%
14%
8%
85%
True Dose-Response Model:Multistage Model:
Linear: P(d) = 1 – exp [ -0.92d]
True Dose-Response Model:Multistage Model:
Linear-Quadratic: P(d) = 1 – exp [ -0.08d + 0.84d2]
True Dose-Response Model:Multistage Model:
Quadratic: P(d) = 1 – exp [ -0.92d2]
True Risk Specific Dose [ RSD (1/100,000) ] Ratio = ----------------------------------------------------------------- Linearized Multistage Model Lower Bound on RSD
Sielken & Associates Consulting, Inc.62
Sielken & Associates Consulting, Inc.63
Experiences Helping Develop More Effective Regulations via Interactions
Between the Public, Universities, Regulated Entities and Regulators
There is a vast collection of examples of
Sielken & Associates Consulting, Inc.64
Thank You