probabilistic risk assessment to inform decision making ... · communication of pra results to the...
TRANSCRIPT
Office of the Science AdvisorRisk Assessment Forum
EPA/100/R-14/003 July 2014 www.epa.gov/raf
U.S. Environmental Protection Agency Office of Research and Development Washington, DC 20460
Official Business Penalty for Private Use $300
PRESORTED STANDARDPOSTAGE & FEES PAID
EPAPERMIT NO. G-35
EPA/100/R‐14/003
ProbabilisticRiskAssessmenttoInformDecisionMaking:
FrequentlyAskedQuestions
July25,2014
U.S.EnvironmentalProtectionAgencyOfficeoftheScienceAdvisor
RiskAssessmentForumProbabilisticRiskAnalysisTechnicalPanel
Washington,D.C.20460
ii
DISCLAIMER ThisdocumenthasbeenreviewedinaccordancewithU.S.EnvironmentalProtectionAgencypolicy.Mentionoftradenamesorcommercialproductsdoesnotconstituteendorsementorrecommendationforuse.
PreferredCitation:
U.S.EnvironmentalProtectionAgency(USEPA).2014.ProbabilisticRiskAssessmenttoInformDecisionMaking:FrequentlyAskedQuestions.EPA/100/R‐09/001B.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.http://epa.gov/raf/prawhitepaper/index.htm.
iii
ThisdocumentwasproducedbyaTechnicalPaneloftheRiskAssessmentForumtosummarizetheRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudiesformanagersandagencyscientisttogainabetterunderstandingoftheprinciplesofprobabilisticriskassessment.Theauthorsdrewontheirexperienceinperformingandinterpretingprobabilisticassessmentstoimprovethemanagementofenvironmentalandhealthhazards.Interviews,presentationsanddialogueswithriskmanagersconductedbytheTechnicalPanelhavecontributedtotheinsightsandrecommendationsinthisFAQdocumentandtheassociatedwhitepaper.
AuthorsRobertHetes,EPAOfficeofResearchandDevelopment(DocumentLead)KathrynGallagher,EPAOfficeofWater(formerlywiththeOfficeoftheScienceAdvisor)MarianOlsen,EPARegion2RitaSchoeny,EPAOfficeofResearchandDevelopment(formerlywiththeOfficeofWater)CynthiaStahl,EPARegion3
ProbabilisticRiskAnalysisTechnicalPanelHalûkÖzkaynak,EPAOfficeofResearchandDevelopment(Co‐Chair)RobertHetes,EPAOfficeofResearchandDevelopment(Co‐Chair)KathrynGallagher,EPAOfficeofWater(Co‐Chair)ChristopherFrey,NorthCarolinaStateUniversity(whileontemporaryassignmentatEPA)
(WhitePaperCo‐Lead)JohnPaul,EPAOfficeofResearchandDevelopment(WhitePaperCo‐Lead)PaskyPascual,EPAOfficeofResearchandDevelopment(WhitePaperCo‐Lead)MarianOlsen,EPARegion2(CaseStudyLead)MichaelClipper,EPAOfficeofSolidWasteandEmergencyResponseMichaelMessner,EPAOfficeofWaterKeeveNachman,currentlyatTheJohnsHopkinsBloombergSchoolofPublicHealthZacharyPekar,EPAOfficeofAirandRadiationRitaSchoeny,EPAOfficeofResearchandDevelopmentCynthiaStahl,EPARegion3DavidHrdy,EPAOfficeofPesticideProgramsJohnLangstaff,EPAOfficeofAirandRadiationElizabethMargosches,EPAOfficeofChemicalSafetyandPollutionPrevention(retired)AudreyGalizia,EPAOfficeofResearchandDevelopmentNancyRios‐Jafolla,EPARegion3DonnaRandall,EPAOfficeofChemicalSafetyandPollutionPreventionKhoanDinh,EPAOfficeofChemicalSafetyandPollutionPrevention(retired)HarveyRichmond,EPAOfficeofAirandRadiation(retired)
RiskAssessmentForumScienceCoordinator
JulieFitzpatrick,EPAOfficeoftheScienceAdvisor
ExternalPeerReviewersScottFerson(Chair),AppliedBiomathematicsAnnetteGuiseppi‐Elie,DuPontEngineeringDaleHattis,ClarkUniversityIgorLinkov,U.S.ArmyEngineerResearchandDevelopmentCenterJohnToll,WindwardEnvironmentalLLC
iv
TABLE OF CONTENTS EXECUTIVESUMMARY.........................................................................................................................................1
FREQUENTLYASKEDQUESTIONS....................................................................................................................2
WhyshouldIcareaboutPRA?Whyisitimportanttoriskmanagers?.......................................................2 Asamanager,whatdoIneedtoknowaboutPRA?.............................................................................................2 HowcouldariskmanagerapproachaPRA?Whatwouldamanagerconsider?....................................2 WhatisPRA?Howdoesitcomparewithcurrentapproaches?......................................................................3 WhatarecommonchallengesfacingEPAdecisionmakers?...........................................................................3 Whatarevariabilityanduncertainty?Whatistheirrelevanceinriskassessmentandrisk
management?............................................................................................................................................................4 HowisriskassessmentconductedcurrentlyatEPA?........................................................................................7 Dohumanhealthriskassessmentandecologicalriskassessmenttreatuncertaintyand
variabilitydifferently?...........................................................................................................................................7 HowdoesEPAtypicallyaddressvariabilityanduncertainty?........................................................................7 HowdoesPRAaddressvariabilityanduncertainty?..........................................................................................9 Howcanenhancedcharacterizationofuncertaintyandvariabilityhelpinformdecisions?...........10 Whatistheimpactofvariabilityanduncertaintyonmakingdecisions?.................................................11 Isuncertaintyandvariabilitylimitedonlytoriskassessment?...................................................................11 Whatkeyquestionsrelatedtouncertaintyandvariabilityareaskedorconsideredbydecision
makers?......................................................................................................................................................................12 Whatisthedesiredpercentileofthepopulationtobeprotected?Bychoosingthispercentile,
whomaynotbeprotected?HowcanaddressinguncertaintyandvariabilityusingPRAhelpinformdecisions?...................................................................................................................................................12
WhataresomeofthelimitationsofandchallengesforPRA?.......................................................................14 WhatisEPA’sexperiencewithPRA?.......................................................................................................................14 WhatareotherEPApublicationsonPRA?............................................................................................................17 WhenshouldoneconsiderusingPRA?...................................................................................................................17 Whatistherightlevelofanalysis?............................................................................................................................18 IfoneisgoingtousePRA,whatfactorsshouldbeconsidered?...................................................................20 WhataretheresourcesneededtoconductaPRA?............................................................................................20 DoesPRArequiremoredatathanconventionalapproaches?......................................................................20 CommunicationofPRAresultstothemanagerandcommunity:doesthepresentationofresults
matter?.......................................................................................................................................................................21 HowcanIgetmoreinformationonPRA?..............................................................................................................21
GLOSSARY..............................................................................................................................................................22
REFERENCES.........................................................................................................................................................26
v
LIST OF FIGURES, TABLES AND EQUATIONS Figures Figure1.Source‐to‐EffectContinuum.............................................................................................................................5
Figure2.IngestionofDrinkingWaterforAdults.......................................................................................................6
Figure3.GeneralPhasesoftheRiskAssessmentProcess.....................................................................................8
Figure4.WhatQuestionsCanPRAAddress?.............................................................................................................13
Figure5.TieredApproachforRiskAssessment.......................................................................................................19
Tables Table1.SelectedExamplesofEPAApplicationsofRiskAssessmentTechniquesfromDeterministic
tomoreComplexAnalysis..................................................................................................................................15
Equations Equation1.DeterministicRiskAssessment.................................................................................................................7 Equation2.ProbabilisticRiskAssessment...................................................................................................................9
1
EXECUTIVE SUMMARY Probabilisticriskassessment(PRA),initssimplestform,isagroupoftechniquesthatincorporatevariabilityanduncertaintyintoriskassessments.Variabilityreferstotheinherentnaturalvariation,diversityandheterogeneityacrosstime,spaceorindividualswithinapopulationorlifestage,whileuncertaintyreferstoimperfectknowledgeoralackofpreciseknowledgeofthephysicalworld,eitherforspecificvaluesofinterestorinthedescriptionofthesystem(USEPA2011c).Variabilityanduncertaintyhavethepotentialtoresultinoverestimatesorunderestimatesofthepredictedrisk.
PRAprovidesestimatesoftherangeandlikelihoodofahazard,exposureorrisk,ratherthanasinglepointestimate.StakeholdersinsideandoutsideoftheAgencyhaverecommendedamorecompletecharacterizationofrisks,includinguncertaintiesandvariability,inprotectingmoresensitiveorvulnerablepopulationsandlifestages.PRAcanbeusedtosupportdecision‐makingriskmanagementbyassessmentofimpactsofuncertaintiesoneachofthepotentialdecisionalternatives.TheaimofthisisdocumentistosummarizetheRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(USEPA2014a)formanagersandagencyscientiststogainabetterunderstandingoftheprinciplesofPRAwithoutthemoredetaileddiscussionpresentedintheWhitePaper.
Numerousadvisorybodies,suchastheScienceAdvisoryBoard(SAB)andtheNationalResearchCouncil(NRC)oftheNationalAcademyofSciences(NAS),havealsorecommendedthatEPAincorporateprobabilisticanalysesintotheAgency’sdecision‐makingprocess.EPA’sRiskAssessmentForum(RAF)formedaTechnicalPanel,consistingofrepresentativesfromtheAgency’sprogramandregionaloffices,todevelopthisFrequentlyAskedQuestions(FAQ)documentanditscompanionpublicationtitledRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(USEPA2014a).TheRAFisalsorecommendingthedevelopmentofAgencyresources,suchasaclearinghouseofPRAcasestudies,bestpractices,resourcesandseminars,toraisegeneralknowledgeabouthowtheseprobabilistictoolscanbeused.
ThepurposeofthisFAQdocumentistopresentgeneralconceptsandprinciplesofPRA,describehowPRAcanimprovethebasesofAgencydecisions,andprovideillustrationsofhowPRAhasbeenusedinriskestimationandindescribingtheuncertaintyindecisionmaking.Thisdocumentisacompaniontothemoredetailedwhitepaper,whichalsoincludesacompendiumof16relevantcasestudiesrangingintheirlevelofcomplexity.Bothdocumentsdescribethepotentialandactualusesofprobabilistictoolsintheriskdecisionprocessandfacilitatetheirimplementationinhuman,ecologicalandenvironmentalriskanalysisandrelateddecision‐makingatEPA.Thedocumentsdonotprescribeaspecificapproachtoriskassessment;rather,theydescribethevariousstagesandaspectsofanassessmentordecisionprocessinwhichprobabilistictoolsmayaddvalue.Thesearenotregulatoryguidancedocuments,buttheyaredesignedtobehelpfulhandbooksand/orreferencesforriskassessors,riskmanagersanddecision‐makersinterestedinimprovingtheriskassessmentanddecision‐makingprocess.
Overall,thisFAQdocumentanswersmorethan20questionscommonlyaskedaboutPRA,beginningwith“WhyshouldIcareaboutPRA,andwhyisitimportanttoriskmanagers?”andendingwith“HowcanIgetmoreinformationonPRA?”ThesequestionscoverawiderangeofPRAissues,andriskmanagers,decision‐makersandotherinterestedreaderswillfindaplethoraofPRAinformationinthisdocument.Furtherdetailedinformation,includingrecommendationsforfutureactivities,areprovidedinthecompanionwhitepaper(USEPA2014a).
2
FREQUENTLY ASKED QUESTIONS
Why should I care about PRA? Why is it important to risk managers? Uncertaintyisunavoidable,evenwhenusinghighlyaccuratedatawiththemostsophisticatedmodels.EPAmakesdecisionsinthepresenceofuncertainty.Therefore,understandingthedecisioncontextiscriticalforAgencydecisionmaking.VariousstakeholdersinsideandoutsideoftheAgencyhaverequestedamorecomprehensivecharacterizationofrisks,includinguncertainties,inprotectingmoresensitiveorvulnerablepopulationsandlifestages.Probabilisticriskassessment(PRA)isonewaytocharacterizetheuncertaintyassociatedwithanyriskassessment.Aspartofadecisionanalysis,theenhanceduseofPRAandcharacterizationofuncertaintywouldallowEPAdecision‐makersopportunitiestoassessuncertaintypertainingtoitseffectondecisionsandexplorethedefensibilityoftheavailableriskmanagementoptionsthroughamorerobustandtransparentprocess.Mostoften,riskmanagerswanttoknowifbetterunderstandingofuncertaintiesmightsupportadifferentdecisionalternativeorprovidefurthersupportfortheselecteddecision.
TheuseofPRAinaddressinguncertaintyandvariabilityatEPAisnotsystematicallypracticed,andthelackthereofisoftenamajorrecommendationfrombothinternalandexternalreviewsofEPAproductsandprocedures(e.g.,theSABreviewofEPApracticesin2007[USEPA2007b],theNRCreviewoftheDioxinReassessment[NRC2006],theOfficeofManagementandBudget’sCircularA‐4[OMB2003]andUpdatedPrinciplesforRiskAnalysis[OMB2007]).TheAgencyhaspublishedbasicguidanceaswellassomeprogram‐specificproceduresandapplicationsforPRA(USEPA2001).AsthescienceofPRAcontinuestoevolve,theenhanceduseofthisapproachwillfacilitatebettercharacterizationofuncertaintyandimprovetheoveralltransparencyandqualityofEPAassessments.PRAapproachesprovideadditionaltoolstoaddressspecificchallengesfacedbymanagersandimproveconfidenceinAgencydecisions.PRAcaninformdecision‐makersaboutspecificsegmentsofthepopulationwhoareatrisk,asopposedtothepopulationasawhole.APRAalsocanaddconfidencetotheconclusionsofadeterministicriskestimate.Thisinformationcanbeimportanttomanagerswhenadifferentdecisionmightbemadeiftheupperorlowerlimitsoftherangeofestimatedexposures,dosesorriskswerepresentedinananalysisasdescribedmorefullyinRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(EPA2014a).
As a manager, what do I need to know about PRA? Thereareprobability,uncertaintyandvariabilityinmanyfields,suchasweatherforecasting,politicalpollsorclimatechangepredictions.PRAcanbeusedineachoftheseexamplesandothers,atmanylevelsordegreesofsophistication,tosupportorimprovedecisionsinavarietyofdifferentdecisioncontexts.ThefollowingsectionsdescribethebasicsofPRAinmoredetail,howitcanbeusedtosupportdecisions,andwhattoconsiderwhenpursuingaPRAorinterpretingPRAresults.
How could a risk manager approach a PRA? What would a manager consider? PRAresultsarelikelyjustsomeoftheinformationthatriskmanagersmightusetoinformtheirdecision‐making.Iftheriskmanageristryingtoansweraquestionthatrequirestheuseofmanydifferentcriteria(whereriskmightbejustonecriterion),thentheymightwanttocontemplatetheuseofformaldecisionanalytictechniques.Withdecisionanalytictechniques,riskmanagerscanexploretherelativeimportanceofriskinformationcomparedtootherinformationinmakingthedecision,andexplorehowuncertaintyaffectspotentialdecisions.PRAprovidesinformationorinputtodecisionanalysis.Decisionanalysisisalogicalprocedureforthebalancingoffactorsthatinfluenceadecision.Theprocedureincorporatesuncertainties,values,andpreferencesinabasic
3
structurethatmodelsthedecision(Howard,1966).DecisionanalysisisabroaderactivitythanPRA,andformaldecisionanalytictechniquesareoutsideofthescopeofthisFAQdocumentandcompanionPRAwhitepaper.ThereisarangeofPRAtechniquesthatmightbeusefultosupportenvironmentaldecisions,andtheyvaryincomplexityandresourcerequirements.Communicationbetweentheriskmanagersandriskassessorsiscriticaltoclearlydefinethespecificneedsofthedecision‐makerandthequestionstobeaddressedbytheriskassessment,sothatonecanfocusthePRAontheinformationmostlikelytoinformthedecision.AspartofPlanningandScopingoftheriskassessmenttheriskassessorandriskmanagershouldevaluatethetypesoftechniquesappropriatetomeetthegoalsoftheassessmentandestablishaprocessforcompletingandreviewingthePRAinacost‐effectiveandtimelymanner.Thedialogueshouldcontinuethroughouttheprocess(U.S.EPA2014b).
What is PRA? How does it compare with current approaches? Currentapproachestoriskassessmenttypicallyrelyonsinglepoint(oftencalled“deterministic”)estimatesofrisk,withbriefqualitativedescriptionsofuncertainty.PRAprovidesriskmanagerswithinformationabouttheuncertaintiesinthedata,models,assumptionsandresults.AbasiccharacteristicofPRAisthatitdoesnotgenerateasinglepointestimate(likeconventionalapproaches),butratherproducesalikelihoodandrangethataparticularexposure,doseoreffectwilloccur.PRAisagroupofcomputationaltechniquesthatallowstheanalysisofvariabilityanduncertaintytobeincorporatedintoexposureand/orriskassessments.PRAalsocanbeusedtosupportriskmanagementusingsensitivityanduncertaintyanalysestoassesstheimpactofuncertaintiesoneachofasetofpotentialdecisionalternatives.Asmentionedabove,thistypeofinformationcanbecriticalindecisions,especiallyindeterminingthelikelihoodofexposureorriskinaspecificportionofthepopulation.Unlikeconventionalapproaches,PRAalsoissuitedtoprovidingdecision‐makerswithabetterunderstandingoftheimpactofuncertaintiesontherelativemeritsofeachofthedecisionalternatives,providingtheriskmanagerwithaclearerrationaleforhisorherdecisionchoice.
What are common challenges facing EPA decision makers? ThechallengesfacingEPAdecision‐makersincludetheneedtomakecomplex,science‐baseddecisions,informedbyconsideringtheimpactofuncertainties,whereconflictingstakeholderperspectivesandshorttimeframesareinvolved.Attributescriticaltodecision‐makinginclude:(1)understandingwhomorwhatwillbeprotected;and(2)knowingtheappropriatedegreeofconfidenceintheestimatedprotectionprovidedbyaparticulardecision.Decisionsareoftentime‐sensitiveandneedtobemadebasedonthecurrentstateofknowledge.Theymayinvolvethecomparisonofseveralalternatives.Inmostcases,thehealthandenvironmentalimpactsofenvironmentalexposurescannotbeisolatedandmeasureddirectly;therefore,riskassessmentmethodstoestimatethehealthandecologicalrisksbasedonavailabledataandinformationcanbeused.BecauseEPAdecisionmakersmayfacechallengesindecisionmakingthatarenotcompletelyrelatedtodecidingwhetherornottousePRA,theymustachieveabalanceinthedecision‐makingprocess,tradingoffthecostsandbenefitsofmoredetailedanalyseslikePRAwiththeiroveralldecisiongoals.
Makingariskdecisionmayalsoconsiderotherfactors,suchastheeconomicimpactsorcostsofalternativestoaddressdifferentialrisksinavulnerablepopulation.EPAdecisionmakersneedtoconsidermultipledecisioncriteriainformedbyvaryingdegreesofconfidenceintheunderlyinginformation,understandingtherelationshipbetweenandamongthosedecisioncriteria(includingmulti‐pollutantandmulti‐mediaeffects)anddecisionalternatives.
4
What are variability and uncertainty? What is their relevance in risk assessment and risk management? Variabilityreferstotheinherentnaturalvariation,diversityandheterogeneityacrosstime,spaceorindividualswithinapopulationorlifestage,whileuncertaintyreferstoimperfectknowledgeoralackofpreciseknowledgeofthephysicalworld,eitherforspecificvaluesofinterestorinthedescriptionofthesystem(USEPA2011c).Variabilityanduncertaintyhavethepotentialtoresultinoverestimatesorunderestimatesofthepredictedrisk.
Variabilityreferstonatural,inherentvariation,variabilityisunavoidable,andcannotbereducedinthewayuncertaintycan.Dataandinformationcanbeusedtoprovideabetterdescriptionandunderstandingofvariabilityintheworldoraparticularsystemtherein.Variabilityispresentinallaspectsofthesource‐to‐effectcontinuum(Figure1),includingthefollowing:
Howpollutantsarereleased(e.g.,theeffectivenessofemissioncontrols).
Howpollutantsareinfluencedbyenvironmentalconditionsoncereleased(e.g.,meteorology—temperature,windandprecipitation).
Theexposureofpollutantstoreceptors(e.g.,inhalationoringestionrates).
Theeffectsofpollutants(e.g.,endpoint,healthstatus,geneticsusceptibility).
Anexampleofvariabilityistheamountofwaterconsumedbyanadultpopulation.Forexample,asurveyof1,000peopleondailywaterconsumptionmightresultinthedistributiondepictedinFigure2A.AnalternativepresentationofthesamedistributionusingcumulativeprobabilityisgivenbyFigure2B.Bothfiguresshowthat5percentoftheadultpopulationconsumes2.96litersofwaterperdayormore,whereasthemean(average)individualconsumptionis1.04litersofwaterperday(basedonstudiesidentifiedinthe1997editionoftheExposureFactorsHandbook[USEPA1997a],whichwasupdatedin2011[USEPA2011b]).
Uncertaintyisthelackofunderstandingoftheworld;whileunavoidableitcanbereducedthroughadditionalinvestigationorcollectionofbetterinformation.Numerousschemesforclassifyinguncertaintyhavebeenproposed,mostfocusonthreebroadtechnicalcategories:
Inputorparameteruncertaintyreferstouncertaintiesinspecificestimatesorvaluesusedinamodel,suchastheaveragedrinkingwaterintakerate.
Modeluncertaintyreferstogapsinthescientificknowledgeortheorythatisrequiredtomakeaccuratepredictions,suchashowtwocorrectlyspecifyexposurestowater.
Scenariouncertaintyreferstoerrors,typicallyofomission,resultingfromincorrectorincompletespecificationoftheriskscenariotobeevaluatedsuchaserrorsintheproblemformulationomittingindirectwateringestionexposurepathways.Theriskscenarioisasetofassumptionsforthesituationtobeevaluated.
Adecisionanalysiswouldincludenotonlytheimpactoftheinput,modelandscenariouncertaintiesontherelativeattractivenessofpotentialdecisionalternatives,butalsowouldincludethedegreetowhichspecificchoices(suchasselectinginputdata,modelsandscenarios,andevenhowtheproblemordecisionanalysisisframed)alsomayimpacttherelativeattractivenessofpotentialdecisionalternatives.
Fromariskmanager’sperspective,understandingbothuncertaintyandvariabilityareimportant.Variabilityrelatestoourunderstandingofwhomorwhatweareprotecting,anduncertaintyrelates
5
Figure 1. Source‐to‐Effect Continuum. Variability exists within each aspect of the source‐to‐effect continuum, the path from an environmental source to resulting exposures and ultimate health outcomes (adapted from NRC 2009, Figure 4‐1, page 95).
toourconfidenceintheestimateandthelevelofprotectionaffordedbytheoptions.TheAppendixinthecompaniondocumenttothisFAQdocumenttitledRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(USEPA2013),describesingreaterdetailsomespecificissuesrelatedtothemethodologyforPRA.
WhetheritisappropriateorusefultoconsidervariabilityanduncertaintyindecisionsmadebyEPAprogramsandregionsmaybeaffectedbystatutes,precedentsandinputprovidedbystakeholders.CongressestablisheslegalrequirementsthatgenerallydescribethelevelofprotectivenessthatEPAregulationsmustachieve.Infrequently,Congressimposesspecificriskassessmentorotherrequirementsthatmayminimizetheregulatorynecessitytoaddressuncertaintyorvariability.Individualstatutesidentifyvaryingriskstoevaluateandprotectagainst(e.g.,establishamarginofsafety,protectsensitiveresources,orreduceoverallrisks).Inaddition,theymaymandatedifferentlevelsofprotection(e.g.,protectpublicwelfare,preventunreasonablerisk,reduceoverallrisks,orfunctionwithoutadverseeffects)(USEPA2004b).Thesedifferenceshaveimplicationsforwhetherorhowoneanalyzesorconsidersuncertainty(e.g.,thedegreeofconfidenceinprotection)orvariabilitywithinapopulationorlifestage(e.g.,evaluatingthe“expectedriskorcentralestimateofriskforthespecificpopulations”1vs.risksto“sensitiveorsusceptibleindividualsorgroups”2or“individualmostexposed”3).
The2004EPAstaffpapertitledAnExaminationofEPARiskAssessmentPrinciplesandPractices(USEPA2004b)andthePRAwhitepapertitledRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(USEPA2014a;companionpublicationtothisFAQdocument)describeuncertaintyandvariabilityinsomedetailandcanbereferredtoformoreinformation.
1SafeDrinkingWaterActAmendmentsof1996.42U.S.C.§300g‐1(b)(3)(1996).2CleanAirActAmendmentsof1990.42U.S.C.§108(f)(1)(C)(1990).3CleanAirActAmendmentsof1990.42U.S.C.§112(c)(9)(B)(i)(1990).
6
Figure 2. Ingestion of Drinking Water for Adults. Variability can be demonstrated by the amount of water consumed by a population. In this example, the distribution of daily water consumption for adults (ages > 21 years) is depicted in part A, while the cumulative probability is presented in part B.
7
How is risk assessment conducted currently at EPA? GeneralphasesoftheriskassessmentprocessareillustratedinFigure3.Therearemanysimilaritiesbetweenecologicalandhumanhealthriskassessment,andEPAisworkingtoincreaseintegrationbetweenthesetwoprocesses.
Do human health risk assessment and ecological risk assessment treat uncertainty and variability differently? Humanhealthriskassessment(HHRA)addressestheprotectionofonespecies.Ecologicalriskassessment(ERA)generallyaddressestheprotectionofalargenumberofspecies,usingafewspeciesassurrogatesforhundredstothousandsofspeciespresentintheenvironment.HHRAusuallyaddressestheevaluationofrisktoindividualsorgroupsofsusceptiblehumanpopulationsorlifestages.ERAgenerallyaddressestheevaluationofrisktopopulations(notindividuals,exceptinthecaseofendangeredspecies),communitiesoforganismsorecosystems.HHRAgenerallyfocusesonsublethaleffectsorendpoints.ERAfrequentlyfocusesonlethality,reproductionandgrowthasmajorassessmentendpoints.
How does EPA typically address variability and uncertainty? BecauseEPAcannotperformatime‐andresource‐intensiveriskassessmentforeverysituationanddecision,theAgencymustbestrategicindeterminingwhenmoreintensiveassessmentsareneeded.WhenEPAdoesnotexplicitlyquantifythedegreeofconfidenceinariskestimate,theAgencyattemptstoincreaseconfidencethatriskisnotbeingunderestimatedbyapplyingvariousassumptionstoaddressuncertaintyandvariability.AsdepictedinEquation1,methodsthatrelyonacombinationofpointestimates—someconservative(highparametervaluesthataremorelikelytooverestimaterisk)andsomethataretypicaloraverage—usuallymultiplythepointestimatestoachieveasingleestimate(e.g.,conservativepointestimatesyieldaconservativeestimate,averagepointestimatesyieldanestimateforanaverageortypicalindividual).
Equation 1. Deterministic Risk Assessment
= RISK Concentrationin environment
Exposure Duration
Ingestion or Inhalation Rate
Toxicity Factor
[C] [E] [I] [T]
[c] [e] [i] [t]Central tendency (average) values for all parameters
High-end values for some or all parameters
Risk. .
Yields a reasonable estimate for average or typical individual
Yields estimate that is likely biased high (conservative)
[ [] [ [] ] ]X X X
=
X
X X X
XX =
8
Figure 3. General Phases of the Risk Assessment Process. Risk assessment is an iterative process comprised of planning, scoping and problem formulation; analysis (e.g., hazard identification, dose‐response assessment and exposure assessment); interpretation and risk characterization; and risk communication.
9
Thisapproachtypicallyproducesasingleestimateofrisk(e.g.,a1x10‐6excesscancerrisk,whichisanincreasedriskofonein1millionadditionalcancers)alongwithaqualitativediscussionofuncertainty.Thesearegenerallyreferredtoas“deterministicassessments”or“pointestimates”ofriskbecausealltheelementsofthecalculationareassumedtobedeterminedwithoutvariationatsinglevalues.Pointestimatesareuseful,particularlyinscreeningassessmentswitheithercancerornoncancerendpoints,buttheinherentvariabilityanduncertaintiesarenotquantifiedfully.TheseinherentlimitationscanaffectEPAdecisionsinthefollowingways:
TheinabilitytocharacterizeexplicitlythebasicelementsofEPAdecisions—whomorwhatisbeingprotectedorwithwhatdegreeofconfidence.
Theinabilitytocapturemorerealisticallytheriskrangeinsteadofapointestimate.
Theinabilitytocompareaccuratelytheoptions(e.g.,cleanuplevels,permitlevels,regulations,actions)duetothedifferinglevelsofconservatismimplicitintheindividualestimates.
Thedecreasedabilitytomaketradeoffsorreachanappropriatebalancebetweenbenefitsandcosts.
Useofapointestimatedrawscriticismandengendersdebateforbeingconservativeandunrealisticorforprovidinginadequateprotection.Thesecriticismsmaynotbeansweredreadilyduetolimitationsineitherthemethodsordataused.
Pointestimatesdonotpermitmeasurementofuncertaintybecausetheyarepresentedasasingle“true”value,ratherthanarangeofpossiblevalues.
How does PRA address variability and uncertainty? PRAusesdistributionsofvaluesthat,forvariability,reflectvariationsintherealworld,andforuncertainty,reflectlackofknowledgeregardingparameters,modelsand/orscenarios.Theresultisanoverallprobabilitystatementoftherisk;forexample,whattherisksaretotheaverageormeanindividualandthehigh‐endindividual,suchasthe95thpercentile,illustratedinEquation2.Thehypotheticalexampleillustratestheapproachforestimatingindividualrisk.Insomecases,exposureandhealthriskareestimatedforanentirepopulationoramongsusceptiblepopulationsandlifestages.
Equation 2. Probabilistic Risk Assessment
95th % population -level risk
% p
opul
atio
n
risk
Mean risk95th %
population -level risk
% p
opul
atio
n
risk
Mean risk95th %
population -level risk
% p
opul
atio
n
risk
Mean risk= RISK Concentration
in environment
Exposure Duration
Ingestion or Inhalation Rate
Toxicity Factor[ [] [ [] ] ]X X X
X X X =
10
PRAcouldbeappliedtooneormorephasesoftheriskassessmentparadigm,includinghazardcharacterizationaswellasexposure,toxicityand/orriskassessment.InthepastEPAhasfocusedprobabilistictechniquesonpredictionsofexposures.Probabilistictechniquescanalsobeusedduringtheeffectsassessmentphasetoaddressvariability/uncertaintyindataandactivitiesthatincludethefollowing:
Hazardidentification;
Assessmentofdose‐responserelationships(e.g.,aprobabilisticreferencedose(RfD)couldhelpreducepotentialinaccuraciesintheestimationofriskclosetoorbelowtheRfD);
Evaluationofhealtheffectsdata;and
Considerationofdifferentialvulnerabilitiesandsensitivitieswithinapopulation.
How can enhanced characterization of uncertainty and variability help inform decisions? Pertainingtoinputparameter,modelandscenariouncertainties,thereareseveralwaysbywhichvarioustypesofPRAcaninformriskmanagementdecisionmaking.First,asensitivityanalysiscandetermineifmorerefinedinformationaboutthedistributionandrangeofdatawouldhaveasubstantialeffectontheassessmentofriskandmanagementoptions.Thisanalysiswillhelpdeterminehowrobustadecisionmaybewithcurrentinformationand/oridentifywhichparametershavingfurtherrefinedinformationwouldmostbenefitthedecision.Second,PRAcanbeusedtodescribeuncertaintyintheriskassessment,increasethetransparencyoftheinputstothedecisionandassistinselectingamongvariousmanagementoptions.Inaddition,PRAcandothefollowing:
EnhanceEPAdecisionsbyprovidingmoreinformationaboutthepossibleimpactsofalternativedecisions.
Provideclarityonwhomorwhatweareprotectingandthedegreeofconfidencethatcanbeplacedontheestimatesofprotectionprovidedbyagivendecision.
Allowforamoredetailedcomparisonofalternativeriskmanagementoptionsintermsoftheestimatedimpactsonbothprotectionandcosts.
Characterizetheinherentuncertaintiesandimpactofthoseuncertaintiesonthedecisionandimprovetheoverallconfidenceinspecificdecisions.
Supporttheunderstandingofthecost,timeandfeasibilityofobtainingimprovedinformationoncriticaluncertainties.
TheNRCrecommendedthatEPAadopta“tiered”approachtoselectthelevelofdetailusedinuncertaintyanalysisandvariabilityassessments(NRC2009).Furthermore,itrecommendedthatadiscussionofthelevelofdetailneededforuncertaintyanalysisandvariabilityassessmentshouldbeanexplicitpartoftheplanning,scopingandproblemformulationphaseoftheriskassessmentprocess.ThecompaniondocumenttitledRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(USEPA2014a)providesadetaileddiscussionofPRAandEPA’sexperiencewithriskassessments,highlightingthestepsintheprocessduringwhichPRAcanbeappliedtocharacterizeuncertaintyandvariability,including:
Planning,scopingandproblemformulation(e.g.,developinganenvironmentalsamplingplanthatconsiderscost,timelinessandtheeliminationofsamplingbias).
Analysis(e.g.,consideringuncertaintyandvariabilityinexposurefactors,exposurescenarios,environmentalconcentrationsofagentsanddose‐responserelationships;
11
assessingwhoisexposed;modelingandevaluatingwhetherfurtherplanning,scopingandproblemformulationisrequired).
Interpretationandriskcharacterization(e.g.,describingvariabilityinriskestimatesbecauseofvariabilityofgeographicconditionsandexposurewithinapopulation,conductinganuncertaintyanalysisbyperformingasensitivityanalysis,estimatinguncertaintyandidentifyingsourcesofuncertainty).
Riskcommunication(e.g.,beingexplicitaboutvariabilityanduncertainty;evaluatingwhetherfurtherriskcharacterizationisrequired).
Decisionmaking(e.g.,prioritizingdataneeds,evaluatingwhetherfurtherriskcommunicationoranalysisisrequired).
What is the impact of variability and uncertainty on making decisions? Variabilityanduncertaintyareinherentinriskassessments.Wheredataandinformationareincompleteorareinadequate,makinginformeddecisionsismoredifficult,andwithoutconsideringuncertaintyinthestateofknowledgeortheimpactofchoicesmade(suchasinselectingdata,models,scenariosorframingtheproblem),thereexistsgreaterpotentialformakingpoorlyinformeddecisionchoices.Inthecaseofenvironmentalregulations,specificdecisionsmayleadtoover‐orunder‐regulationcomparedwithdecisionsthatcouldbemadewithperfectinformation.Settinganenvironmentalstandardthatistoolaxmaythreatenpublichealth,whereasastandardthatisunnecessarilystringentmayimposeasignificanteconomiccostforamarginalgaininpublichealthandenvironmentalprotection.
Theuseofpointestimatesindecisionmakingmightconveyanunfoundedhighdegreeofprecision.Aprobabilisticapproachcanbeappliedratherthandefiningasinglevalueandmightbelesslikelytoimplyundueprecision.Thedisplayofresultsasadistributioncanconveythatnoparticularvaluecarriesspecificrisksofadverseeffects;italsocansupporttheunderstandingofthemagnitudeandlikelihoodofriskofadverseimpactforarangeofexposures,notjustanindividualpointestimateofdoseorexposure.
Is uncertainty and variability limited only to risk assessment? PRAcanalsohelptoinformdecisionsbyforcingdelineationoftheuncertaintypresentandprovidingevaluationmeasuresofoutcomes(oftenreferredtoasadecisionanalysis).Uncertaintiesoftenarerepresentedasprobabilitiesorprobabilitydistributionsingraphsornumericalformats.Thiscanbeveryusefulinevaluatingandcommunicatingtheriskmanagementalternativesavailabletothedecisionmaker.
Indecisionmaking,uncertaintyandvariabilityarenotlimitedtodata,modelsandscenarioconstructionandhowthesemayaffectthepreferenceofonedecisionalternativeoveranother.Adecisionmakermaybefacedwithcombiningmanydifferentdecisioncriteriarequiringtheuseofweightsorvalues(e.g.,whatismoreimportantandbyhowmuch?)tochooseadecisionalternative.Theuseandcombinationofdecisioncriteriamayintroduceotheruncertaintiesandvariability.Thecombinedimpactofdata/model/scenarioselection,expertjudgmentandweighting/valuesonthepreferenceofdecisionalternativesfallsunderthebroadumbrellaofthefieldofdecisionanalysis.
Byfullyunderstandingthewaysinwhichalltypesofuncertaintyinfluencetheinformationusedindevelopingadecision,managerscanbemoreconfidentthattheirdecisionwillproducetheresultsthattheyseekaswellasbebetterabletodefendtheirdecisionsandexplainhowthechosenalternativemeetsAgencyandstakeholdergoals.Thissummaryisfocusedprimarilyonaddressinguncertaintyandvariabilityrelatedtodata/inputs,modelsandscenarios.Uncertaintyand
12
associatedwithdecisioncriteriavaluesandweightsarebeyonditsscope.AlthoughworkinthisaspectofdecisionmakingatEPAhasbeenlimited,someexamplesareavailableatEPA’sMulti‐CriteriaIntegratedResourceAssessment(MIRA)website(http://www.epa.gov/reg3esd1/data/mira.htm).
What key questions related to uncertainty and variability are asked or considered by decision makers? ThePRATechnicalPanelconductedseveraldialogueswithEPAdecisionmakers,askingthemwhatquestionsarisewhentheyarefacedwiththetaskofmakingdecisionsinthepresenceofuncertainty.Thefollowingquestionsrepresenttypicalconcernswhenaddressingdifferenttypesofuncertaintyandvariability:
Wouldmydecisionbedifferentifthedataweredifferent,improvedorexpanded?Wouldadditionaldatacollectionandresearchlikelyleadtoadifferentdecision?Howlongwillittaketocollecttheinformation,howmuchwoulditcost,andwouldtheresultingdecisionbesignificantlyaltered?
Whataretheliabilitiesandconsequencesofmakingadecisionunderthecurrentlevelofknowledgeanduncertainty?
Howdothealternativesandtheirassociateduncertaintyandvariabilityaffectthetargetpopulationorlifestage?
Howrepresentativeorconservativeistheestimateduetodata,methodorscenariouncertainty?
Whatarethemajorgapsinknowledge,andwhatarethemajorassumptionsusedintheassessment?Howreasonablearetheassumptions?
Canaprobabilisticapproach(e.g.,tobettercharacterizevariabilityanduncertainty)beaccomplishedinatimelymanner?
What is the desired percentile of the population to be protected? By choosing this percentile, who may not be protected? How can addressing uncertainty and variability using PRA help inform decisions? PRAcanprovideinformationtodecisionmakersonspecificquestionsrelatedtovariabilityanduncertainty.
Characterizationoftheuncertaintyinestimates(i.e.,whatisthedegreeofconfidenceinanestimate?).Couldtheestimatebeoffbyafactorof2,afactorof10orafactorof1,000?
Criticalparametersandassumptionsthatmostaffectorinfluenceadecisionandtheriskassessment.
“Tippingpoints”wheretheoptionschosenwouldbealterediftheriskestimatesweredifferent,orifadifferentassumptionwasvalid.
Estimatethelikelihoodthatvaluesforcriticalparameterswilloccur.
Testthevalidityofassumptions.
Estimatethedegreeofconfidenceinaparticulardecisionand/orthelikelihoodofspecificdecisionerrors.
Thepossibilityofalternativeoutcomeswithadditionalinformation,orestimatetradeoffsrelatedtodifferentrisksordecisions.
13
PRAcanprovideinformationontheimpactofadditionalinformationondecisionmaking,consideringthecostandtimetoobtaintheinformationandthepotentialforchangeinthedecision(i.e.,thevalueoftheinformation).Forconsiderationofvariability,PRAcanprovidethefollowingtypesofinformationforexposures:
Explicitlydefinedexposuresforvariouspopulationsorlifestages(i.e.,whoarewetryingtoprotect?).Thatis,willtheregulatoryactionkeep50%,90%,99.9%orsomeotherfractionofthepopulationbelowaspecifiedexposure,doseorrisktarget?
Variabilityintheexposures,amongvariouspopulationsorlifestages,andinformationonthepercentileofthepopulationthatisbeingevaluatedintheriskassessment(e.g.,variationsinthenumberoflitersofwaterperkilogram[kg]bodyweightperdayconsumedbythepopulation).Thisinformationishelpfulinaddressing:
TheconservatismofEPA’sriskassessments;
Concernsaboutwhetherparticularexposureswereevaluatedintheriskassessment;
Whomorwhatisbeingprotectedbyamanagementaction;and
Whetherandwhatadditionalresearchmaybeneededtoreduceuncertainty.
AfewhypotheticalexamplesofthetypesofriskassessmentquestionsthatcanbeaddressedexplicitlythroughPRAareillustratedinFigure4.
Figure 4. What Questions Can PRA Address? PRA can be used to determine the likelihood of exposure or risk in a specific fraction of the population.
14
What are some of the limitations of and challenges for PRA? Dataand/orresourcesmaynotbeavailabletosupportprobabilistictechniquesatallstagesinthesameassessment,requiringtheriskassessortocontinuetoapplysomedeterministicscience‐policyassumptionsandconversions.Ifscience‐policyassumptionsordefaultvaluesforparametersareappliedtoanassessmentthatincludesPRA,theyshouldbearticulatedclearlyinthedisseminationofresults.PRAtypicallyrequiresmoretimetodevelopthanadeterministicassessment,butthesetechniquescanfitintoagraduatedortieredapproachtoriskanalysis.
Additionallimitations/challengesincludethefollowing:
PRAmaybemoredataintensivethandeterministicapproaches,requiringadditionaltimeandfinancialandanalyticresourcestoobtainthenecessarystatisticaldistributioninputdataforeachaspectoftheriskassessment.Moreroutineincorporationofprobabilisticdesignsinriskassessmentanditssupportingresearchcouldreducethiscostdifferential.
Todate,PRAtechniqueshavebeenusedbyEPAscientistsmostsuccessfullyintheexposureaspectofhumanhealthriskassessment.AlthoughPRAcanbeusedtocharacterizetheuncertaintyandvariabilityinsituationswithlimiteddata,experienceisneededtousePRAtocharacterizetherangeofeffectsordose‐responserelationshipsforpopulations,includingsensitivepopulationsandlifestages.Moreworkisrequiredtodevelopthesemethodsforothertypesofanalyses.
Thedisseminationofastatisticaldistributionorprobabilityoutputnumbershouldberelatedcarefullytothequalityandcoverageoftheinputdata;otherwise,thePRAresultscouldleadtoafalsesenseofprecision.
PRAassessmentsthatarenottransparentmaybeusedinappropriatelytomanipulateresultsandobfuscatethebasesofdecisions.Itshouldbenotedthatthismayoccurwithdeterministicassessmentsaswell.
What is EPA’s experience with PRA? RecentEPAexperiencewithPRAisprimarilylimitedtoanalyzingdata,modelandscenariouncertaintiesandvariability.Inthepast,however,EPAusually—butnotalways—reliedondeterministicorpointestimatestoevaluaterisk(e.g.,cancerriskof1x10‐6excesscancersoronein1million).TheuseofPRAtoevaluateuncertaintyandvariabilityinriskassessmentsisincreasing.Theseeffortsarevariedacrossprogramofficesandregions,bothincomplexityandinapplications.ManyPRAapplicationsfocusedonspecificelementsofariskassessment(e.g.,exposure),variabilityoruncertainty.ExamplesareprovidedintheAppendixoftheRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(USEPA2014a),thecompanionpublicationtothisFAQdocument.Thewhitepapercontains16casestudies(11HHRAsand5ERAs)thatdetailPRAsthathavebeenconductedtosupportregulatorydecisionsand/orregulatoryimpactanalyses.ExamplesofEPA’suseofdeterministicriskassessmentandPRAareillustratedinTable1,whichhighlights6ofthe16casestudiesinthecompaniondocument.TheseexamplesapplydifferentPRAtoolsinriskassessment.Thecasestudiesrangefromlessresource‐intensiveanalysestomoredetailedandresource‐intensiveapproaches.
U.S.EPAInternalReviewDraft 15 DoNotCite–Quote–orDistribute
Table 1. Selected Examples of EPA Applications of Risk Assessment Techniques from Deterministic to More Complex Analysis
Case Study No. Description Type of Analysis
(Application) Factors Informing
Decisions Type of Risk Assessment
Program/ Region
2*
Atmospheric Deposition to Watershed Contamination: EPA’s Office of Research and Development (ORD) developed an analysis of nitrogen, mercury and polynuclear aromatic hydrocarbons (PAHs) in the Casco Bay Estuary in southwestern Maine.
Deterministic Analysis (Nitrogen Deposition Rate)
Cost-Effectiveness, Timeliness
Ecological ORD
5†
Hudson River Polychlorinated Biphenyl (PCB)-Contaminated Sediment Site: EPA Region 2 evaluated the variability in risks to anglers who consume recreationally caught fish contaminated with PCBs from sediments in the Hudson River.
1-Dimensional Monte Carlo
Analysis (Exposure Factors)
Exposure Variability Within a Population
Human Health
Superfund/
Region 2
(New York)
7†
Environmental Monitoring and Assessment Program (EMAP): ORD developed and the Office of Water (OW) applied probabilistic sampling techniques to evaluate the nation’s aquatic resources under the Clean Water Act (CWA) Section 305(b).
Probabilistic Sensitivity Analysis
(Environmental Sampling Plan)
Representative Results, Cost-Effectiveness, Elimination of Sampling Bias
Ecological ORD/OW
9†
Chromated Copper Arsenate (CCA) Risk Assessment: ORD and the Office of Pesticide Programs (OPP) conducted a probabilistic exposure assessment of children’s exposure (addressing both variability and uncertainty) to arsenic and chromium from contact with CCA-treated wood play sets and decks.
2-Dimensional Monte Carlo
Analysis (Exposure Scenarios)
Exposure Variability, Sensitivity Analysis, Prioritization of Data
Needs
Human Health ORD/OPP
U.S.EPAInternalReviewDraft 16 DoNotCite–Quote–orDistribute
Table 1. Selected Examples of EPA Applications of Risk Assessment Techniques from Deterministic to More Complex Analysis
Case Study No. Description Type of Analysis
(Application) Factors Informing
Decisions Type of Risk Assessment
Program/ Region
13†
Evaluating Ecological Effects of Pesticide Uses: OPP developed a probabilistic model that evaluates acute mortality levels in generic and specific ecological species for user-defined pesticide uses and exposures.
Probabilistic Analysis (Exposure,
Dose-Response)
Variability of Geographic Conditions,
Prioritization of Data Needs
Ecological OPP
14†
Fine Particulate Matter (PM2.5) Health Impacts: ORD and the Office of Air and Radiation (OAR) used expert elicitation to characterize more completely, both qualitatively and quantitatively, the uncertainties associated with the relationship between the reduction in particulate matter (PM2.5) and benefits of reduced PM2.5-related mortality.
Expert Elicitation (Dose-Response)
Estimation of Uncertainty,
Identification of Sources of Uncertainty
Human Health ORD/OAR
* Deterministic risk assessment † Probabilistic risk assessment
U.S.EPAInternalReviewDraft 17 DoNotCite–Quote–orDistribute
What are other EPA publications on PRA? EPA’sexperiencewithPRAincludesnotonlyindividualassessmentsorapplications,butalsothedevelopmentofgeneralguidanceandpoliciesrelatedtodata,modelandscenariouncertainty,suchasthefollowingdocuments:
PolicyforUseofProbabilisticAnalysisinRiskAssessmentattheU.S.EnvironmentalProtectionAgency(USEPA1997c);
GuidingPrinciplesforMonteCarloAnalysis(USEPA1997b);and
RiskAssessmentGuidanceforSuperfund,VolumeIII—PartA,ProcessforConductingProbabilisticRiskAssessment(USEPA2001).
When should one consider using PRA? ItisappropriatetoconsiderPRAanytimethereisanexpectationthatthequantificationofuncertaintyand/orvariabilitymightinfluenceorimpactadecision.AsensitivityanalysisforthedecisioncanhelpmanagersdeterminewhetherhavingsuchinformationiscriticalandwhetherthetimeandresourcesneededforPRAarewarranted.PRAmaynotbeneededwhenthedecisionisroutine,legislativelymandatedorifastandardmethodologyisprescribed.Furthermore,PRAmaynotbeneededwhenthereishighconfidenceinthedataandmodelsusedtosupportthedecision.Planningandscopingdiscussionsareneededbetweentheriskassessorsandtheriskmanagers;thesearedescribedinEPA’sFrameworkforHumanHealthRiskAssessmenttoInformDecisionMaking(USEPA,2014b),EcologicalRiskAssessmentFramework(USEPA1992),GuidelinesforEcologicalRiskAssessment(USEPA1998),andFrameworkforCumulativeRiskAssessment(USEPA2003).AnanalysisplandevelopedduringtheplanningandscopingphasemayindicatethatinformationfromaPRAcouldsignificantlyinformthedecision.Someexamplesare:
Aspecifiedtargetlevelofprotectioninapopulationisidentifiedbythemanager(e.g.,the95thpercentile),anditisnecessarytodemonstratethatthisgoalismet.
Significantequityorenvironmentaljusticeissuesareraisedbyvariationinrisksamongtheexposedpopulationofconcern.
Screening‐levelpointestimatesofriskarehigherthananacceptedlevelofconcern.
Uncertaintyinsomeaspectoftheriskassessmentishigh,anddecisionsarecontentiousorhavelargeresourceimplications.
Specificcriticalriskestimatesandassumptionspointtodifferentmanagementoptions.
ThescientificrigorandqualityoftheassessmentiscriticaltothecredibilityoftheEPAdecision.
Whenascreening‐levelDRAindicatesthatrisksarepossiblyhigherthanalevelofconcernandamorerefinedassessmentisneeded.
Whentheconsequencesofusingpointestimatesofriskareunacceptablyhigh.
Whensignificantequityorenvironmentaljusticeissuesareraisedbyinterindividualvariability.
Whenexploringtheimpactoftheprobabilitydistributionsofthedata,modelandscenariouncertaintiesaswellasvariabilitytogethertocomparepotentialdecisionalternatives.
U.S.EPAInternalReviewDraft 18 DoNotCite–Quote–orDistribute
PRAisusefulespeciallyifthedecisionprocessisiterativeandadaptive,allowingfordetailedandtargetedanalysisovertime.Ifthedecisionhasasingledecisionpointorseparatesinglestepsinmakingadecisionthatwillnotberepeatedorrevised,however,PRAshouldbeusedwithcautionasitmaybeusedtomanipulateresultsifitisnotdesignedproperlyandtransparently.
PRAisnotnecessarilyappropriatewhenregulatoryandlegislativemandatesdefinetheassessment(e.g.,“individualmostexposed”),whichmayminimizestheissueofvariability.Anuncertaintyanalysiswouldstillbeappropriate.
What is the right level of analysis? TheNRC(2009)recommendsatieredapproachtoriskassessmentusingbothqualitativeandquantitative(deterministicandprobabilistic)tools,withthecomplexityoftheanalysisincreasingasprogressismadethroughthetiers.ThewaythatPRAfitsintoagraduatedhierarchical(tiered)approachisdescribedmorefullybelowandisillustratedinFigure5.ThethreetiersinthefigureroughlycorrespondtothethreegroupsofEPAcasestudiesthatusePRAtools,describedinTableA‐1oftheAppendixofthispublication’scompaniondocumenttitledRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(USEPA2014a).
Thereisnosingle“right”levelofanalysis.Instead,theappropriatelevelofanalysisisguidedbythenatureofthedecisionandapplicableregulations.Asisthecaseforriskassessmentingeneral,approachestoPRAandspecificanalyticalmethodsmayvarydramaticallyintheircomplexityandresourceimplications.Theconceptofiterativeortieredanalyses,depictedinFigure5,iswidelyacceptedinriskassessment,andthisappliestoPRAaswell.Highertierscouldreflectincreasingcomplexityandgenerallywillrequiremoretimeandresources.Ananalysistypicallymightstartatalowertier(suchasscreeningforprioritization)andonlyprogressifthereisaneedforamoresophisticatedassessmentcommensuratewiththeimportanceoftheproblem.Highertiersalsoreflectincreasingcharacterizationofvariabilityand/oruncertaintyintheriskestimate,whichmaybeimportantfordecisionmaking.ThereisawiderangeofPRAmethodsandapproachesofvaryingcomplexityandrigorthatcanbeappliedfordifferentpurposes,rangingfromsensitivityanalysistointegratedanalysisofuncertaintyandvariability.Thegoalistochoosealevelofdetailandrefinementforananalysisappropriatetotheoverallobjectivesofthedecisionandtypesofavailabledataandanalysesneededtosupportdecisions.Intheplanningandscopingandproblemformulationphases,itiscriticaltohaveearlyandcontinueddialoguebetweenthemanagerandriskassessortodevelopaclearunderstandingoftheoverallprojectobjectives,theneedsofthedecisionmaker,thetimingofthedecisionandhowPRAmayplayaroleinthedecision‐makingprocess.Theuseofformaldecisionanalyticapproaches,althoughnotoftenusedinthepast,canprovideinsighttodecisionmakersaboutthetypeandlevelofanalysesrequiredtoanswertheirquestions.
U.S.EPAInternalReviewDraft 19 DoNotCite–Quote–orDistribute
Figure 5. Tiered Approach for Risk Assessment. The applicability of a probabilistic approach depends on the needs of decision makers and potential stakeholders. Assessments that are high in complexity and regulatory significance benefit from the application of probabilistic techniques. Source: Adapted from USEPA 2004a and WHO 2008.
Someoftheconsiderationsindeterminingthetypeandlevelofanalysesthatshouldbeperformedinclude:
1. Whetherornottheriskassessmentinitscurrentstateissufficienttodiscriminateamongoptions(i.e.,aclearpathtoexitingtheprocessisavailable).
2. Iftheassessmentisdeterminedtobeinsufficient,whetherornotprogressiontoahigherlevelofcomplexitywouldprovideasufficientbenefittowarranttheadditionaleffortofperformingaPRA.
3. Whethertherearesignificantdifferencesincostsand/orbenefitsbetweenthealternatives.
SomeoftheEPAcasestudiesprovidedintheAppendixofthissummary’scompaniondocumenttitledRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies (USEPA2014a)offerexamplesofhowPRAtoolscanbeusedinaniterativeortieredapproachina
U.S.EPAInternalReviewDraft 20 DoNotCite–Quote–orDistribute
riskassessment.BothCaseStudies1and9dealwithchildren’sexposuretoarsenicinchromatedcopperarsenate(CCA)pressure‐treatedwood.CaseStudy1providesanexampleoftheapplicationofpointestimatesensitivityanalysis(Tier1)toidentifyimportantvariablesforpopulationexposurevariabilityforCaseStudy9.InCaseStudy9,atwo‐dimensionalMonteCarloanalysis(2‐DMCA)wasconductedtoaddressthevariabilityanduncertaintyintheexposureassessment.Twostagesofrandomsamplingwereconductedtodistinguishbetweenvariabilityanduncertaintyinexposureandtoxicityvariables.
If one is going to use PRA, what factors should be considered? IfonedecidesthattheuseofPRAwouldprovidevaluableinformationinsupportofadecision,someotherthingstoconsiderinmovingforwardincludethefollowing:
ResourcesneededtodevelopthePRAandreviewthedocument,including:
ExpertiseofEPAstafftodevelopaPRAorreviewaPRAsubmittedbyacontractorormemberoftheregulatedcommunity.
Timeneededforthedevelopmentandreviewoftheanalyses.
Funding,eitherintramuralorextramural,thatmaybenecessaryfordevelopmentandreviewofthePRA.
Dataavailabilityandformat(e.g.,electronicorpapercopy)fordevelopingdistributionstoincludeinthePRA.
Peerreview,includingeitherinternaland/orexternalreview,whichhastimeandcostimplications.
Communicationofresultstothescientificcommunity,Agencyexecutives,stakeholdersandthepublic.PRAmayoftenincreasethecomplexityoftheriskinformationtobecommunicatedtovariousaudiences.
What are the resources needed to conduct a PRA? PRAgenerallyrequiresmoretime,effortandresourcesthandeterministicassessmentswithstandardscience‐policyassumptions.Theresourceswillvarydependingonthetoolorapproachthatisselected.ThereisacontinuumofPRAmethodologiestochoosefrom,rangingfromsimplesensitivityanalysestocomplexapproachessuchas2‐DMonteCarloAnalysis.Insomecases,simplesensitivityanalyses,whichmayrequirelimitedtimeandriskassessorresources,canbeconducted“in‐house.”MoresophisticatedanalysesmayrequirespecificexpertiseortheuseofspecifictoolsormodelsnotavailableatEPA;maintainingtransparencywillbeimportantintheseefforts.Properapplicationofprobabilisticmethodsrequiresnotonlysoftwareanddata,butalsoguidanceandtrainingforanalystsusingthetoolsaswellasformanagersanddecisionmakerstaskedwithinterpretingandcommunicatingtheresults.Thedevelopmentofstandardizedapproachesand/ormethodscanleadtotheroutineincorporationofPRAinAgencyapproachesandgreatlyreducecostsinfutureapplications.GreaterdevelopmentandunderstandingofthedecisioncontexttowhichpotentialPRAmightcontributewillallowdecisionmakerstomakeamoreinformeddecisionaboutthebenefitsversusthecostsofconductingaPRArelativetotheirdecision‐makinggoals.
Does PRA require more data than conventional approaches? Ingeneral,PRArequiresmoredatathanconventionalapproachesbecausedistributionsofvaluesratherthansinglevaluesareused.Minimumdatarequirementscurrentlyareatopicofdebateinthebroaderriskassessmentcommunity.Minimumdataneedsvarydependingontheanalytical
U.S.EPAInternalReviewDraft 21 DoNotCite–Quote–orDistribute
approachused;empirical‐based(observationalorfrequentist)methodshavesignificantdatarequirementscomparedwithsubjectivemethods.Someofthedatathatwouldbeappliedinafrequentistapproachmaybeavailablealreadyaspartoftheunderlyingdatasetusedinstandarddeterministicanalyses.Asaresult,PRAcanbeappliedinmostcasesprovidedthatthemethodsusedareappropriatefortheavailablebodyofevidenceanddata.
Communication of PRA results to the manager and community: does the presentation of results matter? ThelackoffamiliaritywithPRApresentsachallengeineffectivelypresentingresultstodecisionmakers,stakeholdersandthepublic.ManyviewPRAasahighlytechnicaldisciplinethatusessophisticatedmathematicsandrequiresextensivetrainingtoapplyandunderstand.Singlepointestimatesareeasiertograspformostpeople,basedinpartonfamiliaritywiththisapproachoverthehistoryofEPA.Althoughsomepeopleinitiallyhavedifficultyinterpretingprobabilitydistributionsofvalues,everyonehasacommonbaselineexperiencewithprobability,uncertaintyandvariabilityfromeverydaylife(e.g.,weatherforecasting,oddsofwinningalottery),andthisexperiencecouldbeusedtoframethediscussionofresults.Itisnotnecessarytounderstandtheunderlyingmathematicsoreventoincluderesultsasfulldistributions.Resultscanbedistilleddowntothecriticalessenceordecision‐meaningfulinputofinterest.
Thedifferentaudiencesandtheirrangeofknowledgeandexpertisemustbeconsideredindevelopingmaterialsforeffectivecommunication.WhenadecisionismadetoconductaPRA,itishelpfultoconsiderearlyorientationofthecommunity,managersandothersinthebasicprinciplesbeforethefinaldecisionispresented.Alternatively,itmaybehelpfultopresenttheresultsofthePRAalongwiththepointestimatetoprovideacontextualframefortheresults.
How can I get more information on PRA? ThisdocumentprovidesageneraloverviewandbasicconceptstoestablishsomefamiliarityandafoundationforfurthereducationonPRA.ThecompanionpublicationtothisFAQdocumenttitledRiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies(USEPA2014a)providesadetaileddiscussionofPRAandEPA’sexperiencewithit.TherearenumerousadditionalresourcestoprovidemoredetailaboutPRA,includingEPA’sRiskAssessmentwebsite(www.epa.gov/risk),whichistheAgency’ssourceforlinkstoriskassessmentmethodsandpolicies.
TheRAFcontinuestohaveongoingactivitiesaddressingtheuncertaintyandvariabilityneedsoftheAgency;detailscanbefoundontheRAF’swebsite(www.epa.gov/raf).
U.S.EPAInternalReviewDraft 22 DoNotCite–Quote–orDistribute
GLOSSARY Analysis.Examinationofanythingcomplextounderstanditsnatureortodetermineitsessentialfeatures(WHO2004).
Assessment.InterpretationandevaluationofEnvironmentalMonitoringandAssessmentProgram(EMAP)resultsforthepurposeofansweringpolicy‐relevantquestionsaboutecologicalresources,including(1)determinationofthefractionofthepopulationthatmeetsasociallydefinedvalue,and(2)associationamongindicatorsofecologicalconditionsandstressors(USEPA2010a).
Assessmentendpoint.Anexplicitexpressionoftheenvironmentalvaluethatistobeprotected,operationallydefinedbyanecologicalentityanditsattributes.Forexample,salmonarevaluedecologicalentities;reproductionandageclassstructurearesomeoftheirimportantattributes.Together,salmon“reproductionandageclassstructure”formanassessmentendpoint(USEPA1998).
CumulativeDistributionFunction.Inprobabilitytheoryandstatistics,thecumulativedistributionfunction(CDF)describestheprobabilitythatareal‐valuedrandomvariableXwithagivenprobabilitydistributionwillbefoundtohaveavaluelessthanorequaltox.
Deterministic.Amethodologyrelyingonpoint(i.e.,exact)valuesasinputstoestimaterisk;thisobviatesquantitativeestimatesofuncertaintyandvariability.Resultsalsoarepresentedaspointvalues.Uncertaintyandvariabilitymaybediscussedqualitativelyorsemi‐quantitativelybymultipledeterministicriskestimates(USEPA2006b).
Deterministicriskassessment(DRA).Riskevaluationinvolvingthecalculationandexpressionofriskasasinglenumericalvalueor“singlepoint”estimateofrisk,withuncertaintyandvariabilitydiscussedqualitatively(USEPA2012).
Ecologicalriskassessment.Theprocessthatevaluatesthelikelihoodthatadverseecologicaleffectsmayoccurorareoccurringasaresultofexposuretooneormorestressors(USEPA1998).
Ecosystem.Thebioticcommunityandabioticenvironmentwithinaspecifiedlocationinspaceandtime(USEPA1998).
Environment.Thesumofallexternalconditionsaffectingthelife,developmentandsurvivalofanorganism(USEPA2010a).
Expertelicitation.Asystematicprocessofformalizingandquantifying,typicallyinprobabilisticterms,expertjudgmentsaboutuncertainquantities(USEPA2011a).
Frequentist(orfrequency)probability.Aviewofprobabilitythatconcernsitselfwiththefrequencywith which an event occurs givenalongsequenceofidenticalandindependenttrials(USEPA1997b).
Hazardidentification.Theriskassessmentprocessofdeterminingwhetherexposuretoastressorcancauseanincreaseintheincidenceorseverityofaparticularadverseeffect,andwhetheranadverseeffectislikelytooccur(USEPA2012).
Humanhealthriskassessment(HHRA).1.Theprocesstoestimatethenatureandprobabilityofadversehealtheffectsinhumanswhomaybeexposedtochemicalsincontaminatedenvironmentalmedia,noworinthefuture(USEPA2010b).2.Theevaluationofscientificinformationonthehazardouspropertiesofenvironmentalagents(hazardcharacterization),thedose‐responserelationship(dose‐responseassessment),andtheextentofhumanexposuretothoseagents(exposureassessment).Theproductoftheriskassessmentisastatementregardingtheprobability
U.S.EPAInternalReviewDraft 23 DoNotCite–Quote–orDistribute
thatpopulationsorindividualssoexposedwillbeharmedandtowhatdegree(riskcharacterization)(USEPA2006a).
Inputs.Quantitiesthatareappliedtoamodel(WHO2008).
Model.Amathematicalrepresentationofanaturalsystemintendedtomimicthebehavioroftherealsystem,allowingdescriptionofempiricaldata,andpredictionsaboutuntestedstatesofthesystem(USEPA2006b).
Modeling.Developmentofamathematicalorphysicalrepresentationofasystemortheorythataccountsforallorsomeofitsknownproperties.Modelsoftenareusedtotesttheeffectofchangesofcomponentsontheoverallperformanceofthesystem(USEPA2010a).
MonteCarloanalysis(MCA)orsimulation(MCS).Arepeatedrandomsamplingfromthedistributionofvaluesforeachoftheparametersinagenericexposureorriskequationtoderiveanestimateofthedistributionofexposuresorrisksinthepopulation(USEPA2006b).
One‐dimensionalMonteCarloanalysis(1‐DMCA).Anumericalmethodofsimulatingadistributionforanendpointofconcernasafunctionofprobabilitydistributionsthatcharacterizevariabilityoruncertainty.Distributionsusedtocharacterizevariabilityaredistinguishedfromdistributionsusedtocharacterizeuncertainty(WHO2008).
Parameter.Aquantityusedtocalibrateorspecifyamodel,suchas“parameters”ofaprobabilitymodel(e.g.,meanandstandarddeviationforanormaldistribution).Parametervaluesoftenareselectedbyfittingamodeltoacalibrationdataset(WHO2008).
Probability.Afrequentistapproachconsidersthefrequencywithwhichsamplesareobtainedwithinaspecifiedrangeorforaspecifiedcategory(e.g.,theprobabilitythatanaverageindividualwithaparticularmeandosewilldevelopanillness)(WHO2008).
ProbabilityDensityFunction.Inprobabilitytheory,aprobabilitydensityfunction(pdf)ofacontinuousrandomvariableisafunction,oftendenotedasf(x),thatdescribestherelativelikelihoodforthisrandomvariabletotakeonagivenvalue.
Probabilisticriskanalysis(PRA).Calculationandexpressionofhealthrisksusingmultipleriskdescriptorstoprovidethelikelihoodofvariousrisklevels.Probabilisticriskresultsapproximateafullrangeofpossibleoutcomesandthelikelihoodofeach,whichoftenispresentedasafrequencydistributiongraph,thusallowinguncertaintyorvariabilitytobeexpressedquantitatively(USEPA2012).
Problemformulation.Theinitialstageofariskassessmentwherethepurposeoftheassessmentisarticulated,exposureandriskscenariosareconsidered,aconceptualmodelisdeveloped,andaplanforanalyzingandcharacterizingriskisdetermined(USEPA2004a).
Referenceconcentration(RfC).Anestimate(withuncertaintyspanningapproximatelyanorderofmagnitude)ofacontinuousinhalationexposuretothehumanpopulation(includingsensitivesubgroups)thatislikelytobewithoutanappreciableriskofdeleteriouseffectsduringalifetime.ItcanbederivedfromaNo‐Observed‐Adverse‐EffectLevel(NOAEL),Lowest‐Observed‐Adverse‐EffectLevel(LOAEL),orbenchmarkconcentration,withuncertaintyfactorsgenerallyappliedtoreflectlimitationsofthedataused.ItgenerallyisusedinEPA’snoncancerhealthassessments(USEPA2007a).
Referencedose(RfD).Anestimate(withuncertaintyspanningapproximatelyanorderofmagnitude)ofadailyoralexposuretothehumanpopulation(includingsensitivesubgroups)thatislikelytobewithoutanappreciableriskofdeleteriouseffectsduringalifetime.ItcanbederivedfromaNOAEL,LOAELorbenchmarkdose,withuncertaintyfactorsgenerallyappliedtoreflect
U.S.EPAInternalReviewDraft 24 DoNotCite–Quote–orDistribute
limitationsofthedataused.IttypicallyisusedinEPA’snoncancerhealthassessments(USEPA2011c).
Risk.1.Riskincludesconsiderationofexposuretothepossibilityofanadverseoutcome,thefrequencywithwhichoneormoretypesofadverseoutcomesmayoccur,andtheseverityorconsequencesoftheadverseoutcomesifsuchoccur.2.Thepotentialforrealizationofunwanted,adverseconsequencestohumanlife,health,propertyortheenvironment.3.Theprobabilityofadverseeffectsresultingfromexposuretoanenvironmentalagentormixtureofagents.4.Thecombinedanswersto:Whatcangowrong?Howlikelyisit?Whataretheconsequences?(USEPA2011c).
Riskanalysis.Aprocessforidentifying,characterizing,controllingandcommunicatingrisksinsituationswhereanorganism,system,subpopulationorpopulationcouldbeexposedtoahazard.Riskanalysisisaprocessthatincludesriskassessment,riskmanagementandriskcommunication(WHO2008).
Riskassessment.1.Aprocessintendedtocalculateorestimatetherisktoagiventargetorganism,system,subpopulationorpopulation,includingtheidentificationofattendantuncertaintiesfollowingexposuretoaparticularagent,takingintoaccounttheinherentcharacteristicsoftheagentofconcern,aswellasthecharacteristicsofthespecifictargetsystem(WHO2008).2.Theevaluationofscientificinformationonthehazardouspropertiesofenvironmentalagents(hazardcharacterization),thedose‐responserelationship(dose‐responseassessment),andtheextentofhumanexposuretothoseagents(exposureassessment)(NRC1983).Theproductoftheriskassessmentisastatementregardingtheprobabilitythatpopulationsorindividualssoexposedwillbeharmedandtowhatdegree(riskcharacterization;USEPA2000).3.Qualitativeandquantitativeevaluationoftheriskposedtohumanhealthortheenvironmentbytheactualorpotentialpresenceoruseofspecificpollutants(USEPA2012).
Risk‐baseddecisionmaking.Aprocessthroughwhichdecisionsaremadeaccordingtotheriskeachposedtohumanhealthandtheenvironment(USEPA2012).
Riskmanagement.Adecision‐makingprocessthattakesintoaccountenvironmentallaws,regulations,andpolitical,social,economic,engineeringandscientificinformation,includingariskassessment,toweighpolicyalternativesassociatedwithahazard(USEPA2011c).
Scenario.Asetoffacts,assumptionsandinferencesabouthowexposuretakesplacethataidstheexposureassessorinevaluating,estimatingorquantifyingexposures(USEPA1992).Scenariosmightincludeidentificationofpollutants,pathways,exposureroutesandmodesofaction,amongothers.
Sensitivityanalysis.Theprocessofchangingonevariablewhileleavingtheothersconstanttodetermineitseffectontheoutput.Thisprocedurefixeseachuncertainquantityatitscrediblelowerandupperbounds(holdingallothersattheirnominalvalues,suchasmedians)andcomputestheresultsofeachcombinationofvalues.Theresultshelptoidentifythevariablesthathavethegreatesteffectonexposureestimatesandhelpfocusfurtherinformation‐gatheringefforts(USEPA2011b).
Tieredapproach.Referstovarioushierarchicaltiers(levels)ofcomplexityandrefinementfordifferenttypesofmodelingapproachesthatcanbeusedinriskassessment.Adeterministicriskassessmentwithconservativeassumptionsisanexampleofalowerleveltypeofanalysis(Tier0)thatcanbeusedtodeterminewhetherexposuresandrisksarebelowlevelsofconcern.Examplesofprogressivelyhigherlevelsincludetheuseofdeterministicriskassessmentcoupledwithsensitivityanalysis(Tier1),theuseofprobabilistictechniquestocharacterizeeithervariabilityoruncertaintyonly(Tier2),andtheuseoftwo‐dimensionalprobabilistictechniquestodistinguish
U.S.EPAInternalReviewDraft 25 DoNotCite–Quote–orDistribute
betweenbutsimultaneouslycharacterizebothvariabilityanduncertainty(Tier3)(USEPA2004aandWHO2008).
Two‐dimensionalMonteCarloanalysis(2‐DMCA).Anadvancednumericalmodelingtechniquethatusestwostagesofrandomsampling,alsocallednestedloops,todistinguishbetweenvariabilityanduncertaintyinexposureandtoxicityvariables.Thefirststage,oftencalledtheinnerloop,involvesacomplete1‐DMCAsimulationofvariabilityinrisk.Inthesecondstage,oftencalledtheouterloop,parametersoftheprobabilitydistributionsareredefinedtoreflectuncertainty.Theseloopsarerepeatedmanytimesresultinginmultipleriskdistributions,fromwhichconfidenceintervalsarecalculatedtorepresentuncertaintyinthepopulationdistributionofrisk.(WHO2008).
Uncertainty.Uncertaintyoccursbecauseofalackofknowledge.Itisnotthesameasvariability.Forexample,ariskassessormaybeverycertainthatdifferentpeopledrinkdifferentamountsofwaterbutmaybeuncertainabouthowmuchvariabilitythereisinwaterintakeswithinthepopulation.Uncertaintyoftencanbereducedbycollectingmoreandbetterdata,whereasvariabilityisaninherentpropertyofthepopulationbeingevaluated.Variabilitycanbebettercharacterizedwithmoredatabutitcannotbereducedoreliminated.Effortstoclearlydistinguishbetweenvariabilityanduncertaintyareimportantforbothriskassessmentandriskcharacterization,althoughtheybothmaybeincorporatedintoanassessment(USEPA2011c).
Uncertaintyanalysis.Adetailedexaminationofthesystematicandrandomerrorsofameasurementorestimate;ananalyticalprocesstoprovideinformationregardinguncertainty(USEPA2006b).
Valueofinformation.Ananalysisthatinvolvesestimatingthevaluethatnewinformationcanhavetoariskmanagerbeforetheinformationisactuallyobtained.Itisameasureoftheimportanceofuncertaintyintermsoftheexpectedimprovementinariskmanagementdecisionthatmightcomefrombetterinformation(USEPA2001).
Variability.Referstotrueheterogeneityordiversity,asexemplifiedinnaturalvariation.Forexample,amongapopulationthatdrinkswaterfromthesamesourceandwiththesamecontaminantconcentration,therisksfromconsumingthewatermayvary.Thismayresultfromdifferencesinexposure(e.g.,differentpeopledrinkingdifferentamountsofwaterandhavingdifferentbodyweights,exposurefrequenciesandexposuredurations),aswellasdifferencesinresponse(e.g.,geneticdifferencesinresistancetoachemicaldose).Thoseinherentdifferencesarereferredtoasvariability.Differencesamongindividualsinapopulationarereferredtoasinter‐individualvariability,anddifferencesforoneindividualovertimearereferredtoasintra‐individualvariability(USEPA2011c).
U.S.EPAInternalReviewDraft 26 DoNotCite–Quote–orDistribute
REFERENCES4 Howard,R.A.(1966).DecisionAnalysis:AppliedDecisionTheory,ProceedingsoftheFourthInternationalConferenceonOperationalResearch,pp.55‐71.Wiley‐Interscience.
NRC(NationalResearchCouncil).1983.RiskAssessmentintheFederalGovernment:ManagingtheProcess.Washington,D.C.:TheNationalAcademiesPress.
NRC.2006.HealthRisksfromDioxinandRelatedCompounds:EvaluationoftheEPAReassessment.Washington,D.C.:TheNationalAcademiesPress.
NRC.2009.ScienceandDecisions:AdvancingRiskAssessment.Washington,D.C.:TheNationalAcademiesPress.
OMB(OfficeofManagementandBudget).2003.CircularA‐4.Washington,D.C.:OfficeofManagementandBudget.http://www.whitehouse.gov/omb/circulars_a004_a‐4/.
OMB.2007.MemorandumfortheHeadsofExecutiveDepartmentsandAgenciesonUpdatedPrinciplesforRiskAnalysis.September19,2007.M‐07‐24.http://www.whitehouse.gov/sites/default/files/omb/memoranda/fy2007/m07‐24.pdf.
USEPA(U.S.EnvironmentalProtectionAgency).1992.FrameworkforEcologicalRiskAssessment.EPA/630/R‐92/001.Washington,D.C.:USEPA.http://www.epa.gov/raf/publications/framework‐eco‐risk‐assessment.htm.
USEPA.1997a.ExposureFactorsHandbook.EPA/600/P‐95/002Fa‐c.Washington,D.C.:NationalCenterforEnvironmentalAssessment,OfficeofResearchandDevelopment,USEPA.http://cfpub.epa.gov/ncea/cfm/recordisplay.cfm?deid=12464.
USEPA.1997b.GuidingPrinciplesforMonteCarloAnalysis.EPA/630/R‐97/001.Washington,D.C.:RiskAssessmentForum,USEPA.http://www.epa.gov/ncea/pdfs/montcarl.pdf.
USEPA.1997c.PolicyforUseofProbabilisticAnalysisinRiskAssessmentattheU.S.EnvironmentalProtectionAgency.MemorandumfromFredHansen,DeputyAdministrator.Washington,D.C.:SciencePolicyCouncil,USEPA.http://www.epa.gov/spc/pdfs/probpol.pdf.
USEPA.1998.GuidelinesforEcologicalRiskAssessment.EPA/630/R‐95/002F.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.http://www.epa.gov/raf/publications/pdfs/ECOTXTBX.PDF.
USEPA.2000.SciencePolicyCouncilHandbook:RiskCharacterization.EPA/100/B‐00/002.Washington,D.C.:OfficeofSciencePolicy,OfficeofResearchandDevelopment,USEPA.http://www.epa.gov/spc/pdfs/rchandbk.pdf.
USEPA.2001.RiskAssessmentGuidanceforSuperfund:VolumeIII—PartA,ProcessforConductingProbabilisticRiskAssessment.EPA/540/R‐02/002.Washington,D.C.:OfficeofEmergencyandRemedialResponse,USEPA.http://www.epa.gov/oswer/riskassessment/rags3adt/.
USEPA.2003.FrameworkforCumulativeRiskAssessment.EPA/630/P‐02/001F.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.www.epa.gov/raf/publications/pdfs/frmwrk_cum_risk_assmnt.pdf
4AllInternetlinkswerecheckedatthetimeofpublicationofthisdocument.
U.S.EPAInternalReviewDraft 27 DoNotCite–Quote–orDistribute
USEPA.2004a.AirToxicsRiskAssessmentReferenceLibrary.Volume1:TechnicalResourceManual.EPA/453/K‐04/001A.ResearchTrianglePark,NC:OfficeofAirQualityPlanningandStandards,USEPA.http://www.epa.gov/ttn/fera/data/risk/vol_1/title_page_volume_1.pdf.
USEPA.2004b.AnExaminationofEPARiskAssessmentPrinciplesandPractices.EPA/100/B‐04/001.Washington,D.C.:OfficeoftheScienceAdvisor,OfficeofResearchandDevelopment,USEPA.http://www.epa.gov/osa/pdfs/ratf‐final.pdf.
USEPA.2006a.AFrameworkforAssessingHealthRisksofEnvironmentalExposurestoChildren.EPA/600/R‐05/093F.Washington,D.C.:USEPA.http://ofmpub.epa.gov/eims/eimscomm.getfile?p_download_id=459047.
USEPA.2006b.AirToxicsRiskAssessmentReferenceLibrary,Volume3:Community‐ScaleAssessment.EPA/452/K‐06/001C.Washington,D.C.:USEPA.http://www.epa.gov/ttn/fera/risk_atra_vol3.html.
USEPA.2007a.Concepts,MethodsandDataSourcesforCumulativeHealthRiskAssessmentofMultipleChemicals,ExposuresandEffects:AResourceDocument.EPA/600/R‐06/013F.Washington,D.C.:USEPA.http://cfpub.epa.gov/ncea/risk/recordisplay.cfm?deid=190187.
USEPA.2007b.LetterFromM.G.MorganandR.T.Parkin,ScienceAdvisoryBoard,toS.Johnson,USEPA.February28,2007.EPA/SAB‐07/003.Washington,D.C.:ScienceAdvisoryBoard,OfficeoftheAdministrator,USEPA.http://yosemite.epa.gov/sab/sabproduct.nsf/55E1B2C78C6085EB8525729C00573A3E/$File/sab‐07‐003.pdf.
USEPA.2010a.EnvironmentalMonitoringandAssessmentProgramMasterGlossary.LastmodifiedNovember8.http://www.epa.gov/emap2/html/pubs/docs/resdocs/mglossary.html#ee.
USEPA.2010b.HumanHealthRiskAssessmentOnondagaLake,LakeBottomSubsite:SedimentConsolidationArea,Camillus,NY.Washington,D.C.:USEPA.http://www.epa.gov/region2/superfund/npl/onondagalake/healthriskassess.pdf.
USEPA.2011a.ExpertElicitationTaskForceWhitePaper.Washington,D.C.:USEPA.http://www.epa.gov/stpc/pdfs/ee‐white‐paper‐final.pdf.
USEPA.2011b.ExposureFactorsHandbook:2011Edition.EPA/600/R‐09/052F.Washington,D.C.:NationalCenterforEnvironmentalAssessment,OfficeofResearchandDevelopment,USEPA.http://cfpub.epa.gov/ncea/risk/recordisplay.cfm?deid=236252.
USEPA.2011c.IntegratedRiskInformationSystemGlossary.LastmodifiedAugust21.http://ofmpub.epa.gov/sor_internet/registry/termreg/searchandretrieve/glossariesandkeywordlists/search.do?details=&glossaryName=IRIS%20Glossary.
USEPA.2012.WasteandCleanupRiskAssessmentGlossary.OfficeofSolidWasteandEmergencyResponse,USEPA.LastmodifiedDecember24.http://www.epa.gov/oswer/riskassessment/glossary.htm.
USEPA.2014a.RiskAssessmentForumWhitePaper:ProbabilisticRiskAssessmentMethodsandCaseStudies.EPA/100/R‐09/001A.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.
USEPA.2014b.FrameworkforHumanHealthRiskAssessmenttoInformDecisionMaking.EPA/100/R‐14/001.Washington,D.C.:RiskAssessmentForum,OfficeoftheScienceAdvisor,USEPA.http://www.epa.gov/raf/frameworkhhra.htm.
U.S.EPAInternalReviewDraft 28 DoNotCite–Quote–orDistribute
WHO(WorldHealthOrganization).2004.InternationalProgrammeonChemicalSafetyRiskAssessmentTerminology.Geneva:WHO.http://www.who.int/ipcs/methods/harmonization/areas/ipcsterminologyparts1and2.pdf.
WHO.2008.UncertaintyandDataQualityinExposureAssessment.Geneva:TheInternationalProgrammeonChemicalSafety,WHO.http://www.who.int/ipcs/publications/methods/harmonization/exposure_assessment.pdf.