risk assessment and ria, george gray

Center for Risk Science and Public Health

Risk Assessment and RIA

George Gray

Center for Risk Science and Public Health Department of Environmental and Occupational Health

Milken Institute School of Public Health


Risk Assessment In Many Settings

•  Financial analysis

•  Engineered systems

•  Human health risks

•  Medical treatments

•  Environmental/ecological risks


Risk Arises Because of Uncertainty

•  What could happen?

•  How likely is it to happen?

•  How large will the consequences be?

•  What can be done to manage the risk?

Need to use available information to forecast an uncertain future


Risk Assessment versus Risk Analysis

•  Risk Assessment •  Use of scientific information to estimate hazard,

likelihood of occurrence, and consequences •  Quantitatively characterize variability and uncertainty

in risks •  Requires sound, unbiased use of science to give best

estimates of potential risk

•  Risk Analysis •  Part of larger social process of decision making •  Sets key attributes of problem for risk assessment •  Requires information from areas beyond natural

sciences


RIA and Risk Assessment

•  Risk assessment is often used to estimate the benefits in benefit-cost analysis (BCA) and other forms of regulatory impact analysis

•  Benefits addressed by risk assessment may include reductions in morbidity, mortality, or environmental impacts


What is a Benefit?


What Do We Need From Risk Assessment?

•  Understanding of changes in cases (of some specific outcome) with changes in exposure to the causative agent(s)

•  Understanding of changes in other risks that may occur as exposure changes

•  Characterization of the uncertainty in those estimates


Comparing Costs and Benefits

Costs Risk High

Low


The Challenge

•  Risk assessment four many health and safety outcomes grew up to address questions of standard setting (e.g., pesticide residues, air quality standards) and not BCA

•  Regulatory risk assessment is mix of science, science policy and analytic tools

•  Science policy choices appropriate for some settings may not be appropriate for BCA


Actuarial vs Modeled Risks

•  Actuarial Risks •  based on previous experience with the same risk •  predictions can be made with a great deal of

precision •  examples include diseases, auto accidents, etc.

•  Modeled Risks •  based on data and theory not direct observation of

the risk •  predictions subject to considerable uncertainty •  examples include cancer risk from chemicals , global

warming, etc.


What Current Regulatory Risk Assessment Provides

•  Statistical estimates of outcomes and the changes that might occur with an intervention (e.g., road safety, drug side effects)

or •  Estimates of levels of exposure to agents that protect

against adverse effects (ecological, most non-cancer human health effects)

or •  Continuous relationships between exposure and

response designed to be “conservative” and not underestimate risk (potential carcinogens, some epidemiologically-based outcomes)


Science, Science Policy, and Policy

Science: Posi+ve A process Hypothesis

Data Challenge

Policy: Normative Tradeoffs Judgment

Legal constraints Pragmatic

Science Policy: “Trans-‐science”

How to Use Science in the Face of Uncertainty


Science

•  Individual experiments, trials, observations, measurements, etc.

•  Science may be mandated - e.g., •  EU REACH •  US EPA Pesticides •  US FDA Pharmaceuticals

•  Science may be what is available in reports, scientific literature, etc.


Assessment - Science and Science Policy

•  Assessment provides understanding of the potential harms from exposure to agents

•  Safety Assessment •  Identify levels of exposure unlikely to cause harm

(e.g., Acceptable Daily Intake, Tolerable Intake, Reference Dose)

•  Risk Assessment •  Characterize change in probability of outcome with

changes in exposure (e.g., Cancer Slope Factors, rate of drug side effects, occupational injuries)


Science and Science Policy

•  Rarely does one scientific study provide the information needed to answer policy questions •  Inadequate scope •  Conflicting studies •  Extrapolations

•  This means assessments are conducted in a state of scientific uncertainty

•  Science policy guides the choices and assumptions for dealing with uncertainty


Highlighting Science Policy

•  “Risk assessors might be faced with several scientifically plausible approaches (e.g., choosing the most reliable dose-response model for extrapolation beyond the range of observable effects) with no definitive basis for distinguishing among them. The earlier Committee (NRC 1983 (The Red Book)) pointed out that selection of a particular approach under such circumstances involves what it called a science-policy choice. Science policy choices are distinct from the policy choices associated with ultimate decision-making - NRC 1994 Science and Judgment in Risk Assessment p 27 "

•  “Importantly, remember that risk characterization is not just about science. It makes clear that science doesn’t tell us certain things and that science policy choices must be made.” Page 11"


Some Science Policy Choices

•  Which study? •  Epidemiology or toxicology? •  Toxicology (Which species? Which sex? Which

endpoint?) •  How to reconcile conflicting studies?

•  How to estimate exposure? (Measure? Model? Which Model?)

•  How to estimate dose-response?

•  Report single estimate of risk or range to reflect uncertainty?


Different Science Policy Choices

Chemical US EPA Maximum Contaminant Level

WHO Drinking Water Guideline

1,2 Dichloroethane (solvent, intermediate)

5 µg/L 30 µg/L

Dichloromethane (solvent)

5 µg/L 20 µg/L

Cadmium 5 µg/L 3 µg/L


Different Science Policy Choices

Apple Insecticide US EPA MRL

Codex MRL EU MRL

Chlorpyrifos 0.01 ppm 1 ppm 0.5 ppm

Dicofol 10 ppm - 0.02 ppm

Permethrin 0.05 ppm 2 ppm 0.05 ppm


What Does This Mean for Conducting RIAs?

•  Sometimes standard risk assessment methods aren’t very helpful

•  Need to accurately reflect uncertainty and variability – and current regulatory risk assessment methods may not be helpful

•  RIA world recognizes many of these shortcomings but technical progress has been slow


Comparing Costs and Benefits

Costs Risk High

Low


Costs Risk High

Low

“Safe” Level

The Problem with Safety Assessment


Can’t Judge Benefits of Intervention

•  Even if the RfD did identify a threshold…

Exposure

Baseline Exposure

Final Exposure

“Ris

k”

RfD

Value?

Final Exposure

Value?

Baseline Exposure

“High” risk “Low” risk


Predicting Outcomes •  For RIA we want to be able to value (cost) specific outcomes

in estimating benefits of reducing risks

•  For actuarial risks we can often make predictions about the specific outcome of concern (e.g., occupational deaths caused by falls from ladders)

•  Modeled risks – especially chemical risks – do not predict the actual outcome •  Safety assessment – no prediction of which adverse

outcome could occur •  Cancer risk assessment – no prediction of specific type of

cancer (no requirement of concordance)


Uncertainty & Variability

•  Variability •  “…represents heterogeneity or diversity in a well-

characterized population which is usually not reducible through further measurement or study.”

•  Uncertainty •  “…represents ignorance about a poorly characterized

phenomenon which is sometimes reducible through further measurement or study.”


Uncertainty and Variability in Risk

•  Many sources of uncertainty in risk assessment •  Causality •  Generalization of observations from one situation to

another (different workplaces, animals to humans) •  Model uncertainty for estimation of risk and reductions

in risk

•  Many sources of variability in risk assessment too •  Differences in exposure parameters (e.g., food or water

consumption, driving behavior) •  Spatial variability in measurements or observations


Model Uncertainty

•  Cancer Risk from 3.45 ppb formaldehyde in air •  assume breathe 20 cubic meters of air per day •  assume 70 years of exposure •  Assume population of 10,000,000

Model Predicted Lifetime Cancers One-hit 21,000

Multistage <1

Probit 0


Deliberate Conservatism

“as an Agency policy, risk assessment procedures, including default options that are used in the absence of scientific data to the contrary, should be health protective.” “Use of health protective risk assessment procedures as described in these cancer guidelines means that estimates, while uncertain, are more likely to overstate than understate hazard and/or risk.”

U.S. EPA Risk Assessment Forum (2005) Guidelines for Carcinogen Risk Assessment. EPA/630/P-03/001F


Why Does This Matter?

•  Certainty in estimating benefits will vary by risk type and data source

•  Methods that deal with uncertainty with “precaution” may distort risk-based decision making


Principles for Risk Analysis

1.  Agencies should employ the best reasonably obtainable scientific information to assess risks to health, safety, and the environment.

2.  Characterizations of risks and of changes in the nature or magnitude of risks should be both qualitative and quantitative, consistent with available data. The characterizations should be broad enough to inform the range of policies to reduce risks.

3.  Judgments used in developing a risk assessment, such as assumptions, defaults, and uncertainties, should be stated explicitly. The rationale for these judgments and their influence on the risk assessment should be articulated.

4.  Risk assessments should encompass all appropriate hazards (e.g., acute and chronic risks, including cancer and non-cancer risks, to human health and the environment). In addition to considering the full population at risk, attention should be directed to subpopulations that may be particularly susceptible to such risks and/or may be more highly exposed.

5.  Peer review of risk assessments can ensure that the highest professional standards are maintained. Therefore, agencies should develop policies to maximize its use.

6.  Agencies should strive to adopt consistent approaches to evaluating the risks posed by hazardous agents or events.

http://www.whitehouse.gov/sites/default/files/omb/assets/omb/memoranda/fy2007/m07-24.pdf


Circular A4 - OMB Guidance on Regulatory Analysis

•  Your analysis should provide sufficient information for decision makers to grasp the degree of scientific uncertainty and the robustness of estimated probabilities, benefits, and costs to changes in key assumptions.

•  The principles of full disclosure and transparency apply to the treatment of uncertainty. Where there is significant uncertainty and the resulting inferences and/or assumptions have a critical effect on the benefit and cost estimates, you should describe the benefits and costs under plausible alternative assumptions.

http://www.whitehouse.gov/OMB/circulars/a004/a-4.pdf


Summary

•  Risk assessment is an analytic approach that helps inform Regulatory Impact Assessment •  Magnitude of benefits from management options •  Uncertainty in the estimates

•  Current regulatory risk assessment processes are not optimized for RIA •  Not continuous functions •  May not predict outcomes •  Built in “conservative” assumptions of unknown

magnitude •  RIA is pushing the development of more sophisticated

methods of risk assessment

risk assessment and ria, george gray

Government & Nonprofit

risk science

public health risk assessment

cancer risk

challenge risk assessment

bca regulatory risk

risk predictions subject

public health ria

unbiased use of science