gao - 2012 epa challenges remain with integrated risk information system program

7/31/2019 GAO - 2012 EPA Challenges Remain With Integrated Risk Information System Program

1/53

CHEMICALASSESSMENTS

Challenges Remainwith EPAs IntegratedRisk InformationSystem Program

Report to the Ranking Member,Subcommittee on Energy andEnvironment, Committee on Science,Space, and Technology, House ofRepresentatives

December 2011

GAO-12-42

United States Government Accountability Office

GAO


2/53

United States Government Accountability Office

Highlights ofGAO-12-42, a report to theRanking Member, Subcommittee on Energyand Environment, Committee on Science,Space, and Technology, House ofRepresentatives

December 2011

CHEMICAL ASSESSMENTS

Challenges Remain with EPA's Integrated RiskInformation System Program

Why GAO Did This Study

The Environmental ProtectionAgencys (EPA) Integrated RiskInformation System (IRIS) Programsupports EPAs mission to protecthuman health and the environment byproviding the agencys scientificposition on the potential human healtheffects from exposure to variouschemicals in the environment. The

IRIS database contains quantitativetoxicity assessments of more than 550chemicals and provides fundamentalscientific components of human healthrisk assessments.In response to aMarch 2008 GAO report on the IRISprogram, EPA revised its IRISassessment process in May 2009.GAO was asked to evaluate (1)EPAsprogress in completing IRISassessments under the May 2009process and (2) the challenges, if any,that EPA faces in implementing theIRIS program. To do this work, GAO

reviewed and analyzed EPAproductivity data, among other things,and interviewed EPA officials.

What GAO Recommends

GAO recommends, among otherthings, that EPA assess the feasibilityof the established time frames for eachstep in the IRIS assessment processand make changes if necessary,submit for independent review to anentity with scientific and technicalcredibility a plan for how EPA will

implement the National Academiessuggestions, and ensure that currentand accurate information on chemicalsthat EPA plans to assess through IRISis available to IRIS users. EPA agreedwith GAOs recommendations andnoted specific actions it will take toimplement them.

What GAO Found

EPAs May 2009 revisions to the IRIS process have restored EPAs control of theprocess, increased its transparency, and established a new 23-month time framefor its less challenging assessments. Notably, EPA has addressed concernsGAO raised in its March 2008 report and now makes the determination of whento move an assessment to external peer review and issuancedecisions thatwere made by the Office of Management and Budget (OMB) under the prior IRISprocess. In addition, EPA has increased the transparency of the IRIS process bymaking comments provided by other federal agencies during the interagencyscience consultation and discussion steps of the IRIS process available to the

public. Progress in other areas, however, has been limited. EPAs initial gains inproductivity under the revised process have not been sustained. After completing16 assessments within the first year and a half of implementing the revisedprocess, EPA completed 4 assessments in fiscal year 2011. Further, theincrease in productivity does not appear to be entirely attributable to the revisedIRIS assessment process and instead came largely from (1) clearing the backlogof IRIS assessments that had undergone work under the previous IRIS processand (2) issuing assessments that were less challenging to complete. EPA hastaken longer than the established time frames for completing steps in the revisedprocess for most of its less challenging assessments. However, EPA has notanalyzed its established time frames to assess the feasibility of the time frame foreach step or the overall 23-month process. The agencys progress has also beenlimited in completing assessments that it classifies as exceptionally complex and

reducing its ongoing assessments workload. Beyond the 55 ongoing IRISassessments, the backlog of demand for additional IRIS assessments is unclear.With existing resources devoted to addressing its current workload of ongoingassessments, EPA has not been in a position to routinely start new assessments

EPA faces both long-standing and new challenges in implementing the IRISprogram. First, EPA has not fully addressed recurring issues concerning theclarity and transparency of its development and presentation of draft IRISassessments. For example, as part of its independent scientific review of EPAsdraft IRIS assessment of formaldehyde, the National Academies providedsuggestions for improving EPAs development and presentation of draft IRISassessments in general, including that EPA use a standardized approach toevaluate and describe study strengths and weaknesses and the weight ofevidence. EPA announced that it planned to respond to the National Academies

suggestions by implementing changes to the way it develops draft IRISassessments. Given that many of the issues raised by the National Academieshave been long-standing, it is unclear whether any entity with scientific andtechnical credibility, such as an EPA advisory committee, will have a role inconducting an independent review of EPAs planned response to thesuggestions. In addition, EPA has not addressed other long-standing issuesregarding the availability and accuracy of current information to users of IRISinformation, such as EPA program offices, on the status of IRIS assessments,including when an assessment will be started, which assessments are ongoing,and when an assessment is projected to be completed.ViewGAO-12-42. For more information,

contact David C. Trimble at (202) 512-3841 [email protected].
http://www.gao.gov/products/GAO-12-42http://www.gao.gov/products/GAO-12-42http://www.gao.gov/products/GAO-12-42http://www.gao.gov/products/GAO-12-42http://www.gao.gov/products/GAO-12-42http://www.gao.gov/products/GAO-12-42mailto:[email protected]:[email protected]:[email protected]://www.gao.gov/products/GAO-12-42http://www.gao.gov/products/GAO-12-42


3/53

Page i GAO-12-42 Chemical Assessments

Letter 1

Background 4

EPAs Progress in Completing Assessments under Its Revised

Process Has Been Limited 10

EPA Faces Long-standing and New Challenges in Implementing the

IRIS Program 19

Conclusions 27

Recommendations for Executive Action 28

Agency Comments and Our Evaluation 29

Appendix I Scope and Methodology 31

Appendix II Status of Chemicals in the IRIS Assessment Development

Process, as of September 30, 2011 33

Appendix III Information on Chemicals of Key Concern 34

Appendix IV Projected Completion Dates for IRIS Assessments Currently in the

Assessment Development Process, as of September 30, 2011 41

Appendix V Comments from the Environmental Protection Agency 42

Appendix VI GAO Contact and Staff Acknowledgments 47

Tables

Table 1: EPAs May 2009 IRIS Assessment Process Steps and

Established Time Frames 8

Table 2: Completed IRIS Assessments from May 2009 through

September 2011 14

Table 3: Ongoing IRIS Assessment Workload Balance 15

Contents


4/53

Page ii GAO-12-42 Chemical Assessments

Table 4: May 2009 IRIS Process Step Time Frames and Actual

Completion Times for Each Step 18

Figures

Figure 1: National Academies Risk Assessment Model Used by

EPA 5

Figure 2: Number of Completed IRIS Assessments, Fiscal Years

2002-2011 12

Abbreviations

ATSDR Agency for Toxic Substances and Disease RegistryBBP butyl benzyl phthalateBOSC Board of Scientific CounselorsCEQ Council on Environmental QualityCERCLA Comprehensive Environmental Response, Compensation,

and Liability Act of 1980DBP dibutyl phthalateDEHA di(2-ethylhexyl) adipateDEHP di(2-ethylhexyl) phthalateDIBP diisobutyl phthalateDINP diisononyl phthalateDIPE diisopropyl etherDOD Department of Defense


5/53

Page iii GAO-12-42 Chemical Assessments

DPP dipentyl phthalateEGBE ethylene glycol monobutyl etherEPA Environmental Protection AgencyETBE ethyl tertiary butyl etherFDA Food and Drug AdministrationHHS Department of Health and Human ServicesIRIS Integrated Risk Information SystemIRISTrack IRIS Substance Assessment Tracking SystemMTBE methyl tert-butyl etherNASA National Aeronautics and Space AdministrationNCEA National Center for Environmental AssessmentNCEH National Center for Environmental HealthNIOSH National Institute for Occupational Safety and HealthNTP National Toxicology ProgramOIRA Office of Information and Regulatory AffairsOMB Office of Management and BudgetORD Office of Research and DevelopmentPAH polycyclic aromatic hydrocarbonPART Program Assessment Rating ToolPBPK physiologically based pharmacokineticPCBs polychlorinated biphenylsPFOA perfluorooctanoic acid-ammonium salt

PFOS perfluorooctane sulfonate-potassium saltRDX hexahydro-1,3,5-trinitro-1,3,5-triazineRfC inhalation reference concentrationRfD oral reference doseTAEE tertiary amyl ethyl etherTAME tertiaryamylmethyl etherTCDD 2,3,7,8-tetrachlorodibenzo-p-dioxinTCE trichloroethyleneTCP trichloropropaneTHF tetrahydrofuranTSCA Toxic Substances Control Act

This is a work of the U.S. government and is not subject to copyright protection in theUnited States. The published product may be reproduced and distributed in its entiretywithout further permission from GAO. However, because this work may containcopyrighted images or other material, permission from the copyright holder may benecessary if you wish to reproduce this material separately.


6/53

Page 1 GAO-12-42 Chemical Assessments

United States Government Accountability OfficeWashington, DC 20548

December 9, 2011

The Honorable Brad MillerRanking MemberSubcommittee on Energy and EnvironmentCommittee on Science, Space, and TechnologyHouse of Representatives

Dear Mr. Miller:

The Environmental Protection Agencys (EPA) Integrated RiskInformation System (IRIS) Program supports EPAs mission to protecthuman health and the environment by providing the agencys scientificposition on the potential human health effects that may result fromexposure to various chemicals in the environment. IRIS was created in1985 to help EPA develop consensus opinions within the agency aboutthe health effects from chronic exposure to chemicals, and its importancehas increased over time. The IRIS database contains quantitative toxicityassessments of the health effects of more than 550 chemicals andprovides fundamental scientific componentsqualitative hazardidentification and quantitative dose-response assessmentof humanhealth risk assessments.1 EPAs IRIS Program develops new IRIS

assessments and, as needed, updates existing IRIS values contained inthe IRIS database. These IRIS assessments, in turn, provide scientificinput for risk management decisions, such as whether EPA shouldestablish air or water quality standards to protect the public fromexposure to toxic chemicals or set cleanup standards for hazardouswaste sites. Consequently, IRIS assessments are a critical component ofEPAs capacity to support scientifically sound decisions, policies, andregulations. State and local environmental programs and someinternational regulatory bodies also rely on IRIS for managing theirenvironmental protection programs.

1As we have previously reported, the Toxic Substances Control Act (TSCA) requires EPAto demonstrate that certain health or environmental risks are likely before it can requirecompanies to test the approximately 700 new chemicals introduced into commerceannually or take action to control unreasonable risks by placing restrictions on the tens ofthousands of chemicals already in the agencys TSCA inventory.


7/53


In 2008, we reported that the IRIS database was at serious risk ofbecoming obsolete because the agency had not been able to keep itsexisting assessments current, decrease its ongoing assessmentsworkload to a manageable level, or complete assessments of the mostimportant chemicals of concern.2 In addition, we reported that as of

December 2007, most of the ongoing assessments being conducted atthat time had been in process for more than 5 years and that someassessments of key chemicalschemicals that are likely to cause canceror other significant health effectshad been in process even longer. Forexample, the formaldehyde and dioxin assessments had been ongoingfor 11 and 17 years, respectively.3 We also reported that new Office of

Management and Budget (OMB)-required reviews of IRIS assessmentsby OMB and other federal agenciescalled interagency reviewswereconducted in a manner that limited the transparency and credibility of theassessments and hindered EPAs ability to manage the IRIS assessmentprocess. Because of these issues, we recommended that EPA revise itsIRIS assessment process to develop the timely chemical risk informationthe agency needs to effectively conduct its mission and to better ensurethe development of transparent, credible chemical assessments.

EPA issued a revised process in April 2008 that we concluded, intestimony before Congress, would further exacerbate the timeliness and

credibility concerns we had identified.

4

Because the agency had notdeveloped sufficient chemical assessment information to limit publicexposure to many chemicals that may pose substantial health risksand,in particular, because of EPAs lack of responsiveness to our March 2008recommendationsin January 2009, we added EPAs processes forassessing and controlling toxic chemicals to our list of areas at high risk

2GAO, Chemical Assessments: Low Productivity and New Interagency Review ProcessLimit the Usefulness and Credibility of EPAs Integrated Risk Information System,GAO-08-440(Washington, D.C.: Mar. 7, 2008).

3

Formaldehyde is a gas widely used in such products as pressed wood, paper,pharmaceuticals, leather goods, and textiles. In addition, the term dioxin applies to afamily of chemicals that are often the byproducts of combustion and other industrialprocesses. Complex mixtures of dioxins enter the food chain and human diet throughemissions into the air that settle on soil, plants, and water. For more information onformaldehyde and dioxin, see appendix III.

4GAO, Toxic Chemicals: EPAs New Assessment Process Will Increase Challenges EPAFaces in Evaluating and Regulating Chemicals.GAO-08-743T(Washington, D.C.: Apr.29, 2008).
http://www.gao.gov/products/GAO-08-440http://www.gao.gov/products/GAO-08-440http://www.gao.gov/products/GAO-08-743Thttp://www.gao.gov/products/GAO-08-743Thttp://www.gao.gov/products/GAO-08-743Thttp://www.gao.gov/products/GAO-08-743Thttp://www.gao.gov/products/GAO-08-440


8/53


for waste, fraud, abuse, and mismanagement or in need of broad-basedtransformation.5

In response to our 2008 report and subsequent high-risk designation,EPA revised its IRIS assessment process in May 2009 to, among otherthings, restore EPAs control of the process and increase itstransparency. Our biennial review of high-risk areas in 2011 concludedthat the EPA Administrator needs to continue to demonstrate a strongcommitment to and support of the IRIS Program to ensure that EPAs2009 reforms are implemented effectively and that the program canroutinely provide timely, transparent, and credible assessments.6 In this

context, you asked us to review EPAs IRIS assessment process. Ourobjectives were to evaluate (1) EPAs progress in completing IRISassessments under the May 2009 process and (2) the challenges, if any,that EPA faces in implementing the IRIS program.

In conducting our work, we analyzed EPAs May 2009 assessmentprocess; data from fiscal year 1999 through September 30, 2011, on IRISproductivity, such as the number of IRIS assessments initiated andcompleted; the status of IRIS assessments that are currently in progress;and EPAs goals for completing assessments. To assess the reliability ofthe data, we conducted interviews and e-mail exchanges with EPA

officials about the data system, the method of data input, and internaldata controls and documentation, among other things. We found the datato be sufficiently reliable for the purposes of our report. We alsointerviewed officials from EPAs Office of Research and Developments(ORD) National Center for Environmental Assessment (NCEA), whichmanages the IRIS Program. In addition, we interviewed officials fromother federal agencies involved in the IRIS processincluding OMB andthe Department of Defense (DOD)and groups that have knowledge ofthe IRIS Program. We did not evaluate the scientific content or quality ofIRIS assessments, but we reviewed the suggestions to EPA in peerreview reports on overall improvements to the development of IRISassessments and information on other issues affecting the IRIS Program.

5GAO, High-Risk Series: An Update,GAO-09-271(Washington, D.C.: Jan. 22, 2009).This high-risk area addresses EPAs implementation of the IRIS program as well asimplementation of the Toxic Substances Control Act (TSCA).

6GAO, High-Risk Series: An Update,GAO-11-278(Washington, D.C.: Feb. 16, 2011). Wealso concluded that the EPA Administrator needed to continue to support the agencysTSCA initiatives.
http://www.gao.gov/products/GAO-09-271http://www.gao.gov/products/GAO-09-271http://www.gao.gov/products/GAO-09-271http://www.gao.gov/products/GAO-11-278http://www.gao.gov/products/GAO-11-278http://www.gao.gov/products/GAO-11-278http://www.gao.gov/products/GAO-11-278http://www.gao.gov/products/GAO-09-271


9/53


A more detailed description of our scope and methodology is presented inappendix I.

We conducted this performance audit from July 2010 to December 2011in accordance with generally accepted government auditing standards.Those standards require that we plan and perform the audit to obtainsufficient, appropriate evidence to provide a reasonable basis for ourfindings and conclusions based on our audit objectives. We believe thatthe evidence obtained provides a reasonable basis for our findings andconclusions based on our audit objectives.

This section discusses EPAs risk assessment and risk managementpractices and the May 2009 IRIS process.

EPAs IRIS Program is an important source of information on healtheffects that may result from exposure to chemicals in the environment. Asfigure 1 shows, the toxicity assessments in the IRIS database fulfill thefirst two critical steps of the risk assessment processprovidingqualitative hazard identification and dose-response assessment (seedefinitions below).7 IRIS information can then be used with the results of

exposure assessments (typically conducted by EPAs program or regionaloffices) to provide an overall characterization of the public health risks fora given chemical in a given situation. EPA defines a risk assessment, inthe context of human health, as the evaluation of scientific information onthe hazardous properties of environmental agents (hazardcharacterization), the dose-response relationship (dose-responseassessment), and the extent of human exposure to those agents(exposure assessment). In final form, a risk assessment is a statementregarding the probability that populations or individuals so exposed will beharmed and to what degree (risk characterization). The development ofrisk assessments is directly dependent on the development of toxicityassessments such as those developed by the IRIS Program.

7EPA uses the model presented by the National Academies in Science and Decisions:Advancing Risk Assessment(Washington, D.C.: The National Academies Press, 2009).This publication is also known as the Silver Book.

Background

Risk Assessment and RiskManagement


10/53


Figure 1: National Academies Risk Assessment Model Used by EPA

Note: The National Academies comprise four organizations: the National Academy of Sciences, theNational Academy of Engineering, the Institute of Medicine, and the National Research Council.

A typical IRIS toxicity assessment is based on two sequential analyses:

qualitative hazard identification and quantitative dose-responseassessment. Among other things, a hazard identification identifies healthhazards that may be caused by a given chemical at environmentallyrelevant concentrations; this identification describes the potentialnoncancer and cancer health effects of exposure to a chemical thatresearch studies have suggested or determined. For cancer effects, EPAdescribes the carcinogenic potential of a chemical in a narrative whichincludes one of five weight-of-the-scientific-evidence descriptors, rangingfrom carcinogenic to humans to not likely to be carcinogenic tohumans. The second analysis is the dose-response assessment, whichcharacterizes the quantitative relationship between the exposure to achemical and the resultant health effects; this assessment describes the

magnitude of hazard for potential noncancer effects and increased cancerrisk resulting from specific exposure levels to a chemical or substance.The quantitative dose-response analysis relies upon credible researchdata, primarily from either animal (toxicity) or human (epidemiology)studies. The noncancer dose-response assessments may include


11/53


an oral reference dose (RfD)an estimate of the daily oral exposureto a chemical that is likely to be without an appreciable risk ofdeleterious effects during a persons lifetimeexpressed in terms ofmilligrams per kilogram per day and

an inhalation reference concentration (RfC)an estimate of the dailyinhalation exposure to a chemical that is likely to be without anappreciable risk of deleterious effects during a persons lifetimeexpressed in terms of milligrams per cubic meter.

The focus of IRIS toxicity assessments has been on the potential health

effects of long-term (chronic) exposure to chemicals. According to OMB,EPA is the only federal agency that develops qualitative and quantitativeassessments of both cancer and noncancer risks of exposure tochemicals, and EPA does so largely under the IRIS Program.8

The risk characterization information, which is derived from toxicity andexposure assessmentsexposure assessments identify the extent towhich exposure actually occurscan be used to make risk managementdecisions designed to protect public health. For example, IRISassessments support scientifically sound decisions, policies, andregulations under such key statutes as the Clean Air Act, the SafeDrinking Water Act, and the Clean Water Act, as well as for settingSuperfund cleanup standards of hazardous waste sites.9 Risk

management, as opposed to risk assessment, involves integrating therisk characterization information with other informationsuch aseconomic information on the costs and benefits of mitigating the risk,technological information on the feasibility of managing the risk, and theconcerns of various stakeholdersto decide when actions to protectpublic health are warranted. More specifically, an initial risk managementdecision would be to determine whether the health risks identified in achemical risk assessment warrant regulatory or other actions. As a result,

8OMB, Fiscal Year 2006 Program Assessment Rating Tool (PART) assessment of EPAsHuman Health Risk Assessment Program.

9The Comprehensive Environmental Response, Compensation, and Liability Act of 1980(CERCLA) requires federal agencies to respond to certain releases or threatened releasesof hazardous substances on lands they administer. CERCLA also created a trust fundthe Superfundto provide for certain cleanup activities at nonfederal sites. UnderCERCLA, EPA established the Superfund program to address the threats thatcontaminated sites pose.


12/53


the development of IRIS assessments is of key interest to stakeholders,such as other federal agencies and their contractors, chemicalcompanies, and others who could be affected if regulatory actions weretaken. That is, stakeholders could face increased cleanup costs and otherlegal liabilities if EPA issued an IRIS assessment for a chemical thatresulted in a risk management decision to regulate the chemical to protectthe public.

EPAs process for developing IRIS assessmentsestablished in May2009consists of seven steps. In announcing its revised process in May

2009, EPA noted that the new process would ensure that the majority ofassessments would be completed within 2 years (23 months)asignificantly shorter time than the estimated completion time frame ofabout 6 to 8 years under the previous process. We note that the sevensteps are preceded by a literature search and data call-in, which is notincluded as part of the process or its time frames. Results of the literaturesearch are posted on the IRIS website and announced in the FederalRegister, along with a request for informationthe data call-inaboutany pertinent studies not listed. According to EPA officials, the literaturesearch and data call-in are not part of the process because the agencydoes not dedicate full-time staff to them. EPA officials told us that afterthe literature search, they place IRIS assessments in one of threecategoriesstandard, moderately complex, or exceptionally complex10

on the basis of such factors as the number of available scientific studieson the chemical, the number of potential health effects identified in thesestudies, the staff resources required to complete the assessment, and thelevel of stakeholder interest. However, this process, as written, does notdistinguish among different types of assessments with varying complexityTable 1 outlines the steps in the IRIS assessment process, along with theplanned time frames established by EPA.

10EPA refers to a standard assessment as a tier 1 assessment, a moderately complexassessment as tier 2, and an exceptionally complex assessment as tier 3.

The May 2009 IRIS Process


13/53


14/53


panels.11 These peer reviews typically require more planning and take

longer than the reviews for less complicated assessments.12 Peer reviews

for all other assessments are typically conducted by expert panels thatare independently assembled by an EPA contractor. All panel members,including Science Advisory Board and National Academies panels, arecomposed of individuals with expertise in various scientific and technicaldisciplines who retain their primary involvement in academia, industry,state government, and environmental organizations.

As we reported in 2008, an overarching factor that can affect EPAs abilityto complete IRIS assessments in a timely manner is the compoundingeffects of delays.13 Once a delay in the assessment process occursfor

example, suspending work on an assessment to wait for additionalstudieswork that has been completed can become outdated,necessitating rework of some or all of the steps in the assessmentprocess. Even a single delay can have far-reaching, time-consumingconsequences, in some cases requiring that the assessment processessentially start over.14

11

Congress established EPAs Science Advisory Board in 1978 and gave it a broadmandate to advise the agency on technical matters. The Science Advisory Boardspreliminary work is done by subcommittees or panels focused on various environmentalscience topics. These groups are chaired by Science Advisory Board members, and theirrecommendations are transmitted to the Science Advisory Board for discussion anddeliberation. Recommendations are forwarded to EPA only if the Science Advisory Boarddetermines that it is appropriate.

12Since May 2009, 1 assessment has been externally peer reviewed by the NationalAcademies (formaldehyde), 3 have been externally peer reviewed by the ScienceAdvisory Board (trichloroethylene [TCE], Polycyclic aromatic hydrocarbon [PAH] mixtures,and Tetrachlorodibenzo-p-dioxin, 2,3,7,8-[dioxin]), and 14 have been externally peerreviewed by expert panels that are independently assembled by an EPA contractor(dichloromethane, hexachloroethane, urea, hexavalent chromium, halogenated platinumsalts, 1,4-dioxane [oral route], pentachlorophenol, ethyl tertiary butyl ether [ETBE],hydrogen cyanide, tetrachloroethane-1,1,2,2, dichloroethylene -1,2-cis-, dichloroethylene -

1,2-trans-, trichloroacetic acid, and chloroprene).

13GAO-08-440.

14The National Academies Silver Book observes that delays in the process of assessingrisks may increase overall exposure to risk when decisions are delayed. The Silver Booknotes that the design of a risk-assessment process should balance the pursuit ofindividual attributes of technical quality in the assessment and the competing attribute oftimeliness of input into decision-making. Science and Decisions: Advancing Risk

Assessment.
http://www.gao.gov/products/GAO-08-440http://www.gao.gov/products/GAO-08-440http://www.gao.gov/products/GAO-08-440http://www.gao.gov/products/GAO-08-440


15/53


16/53


May 2009, OMB managed these steps, and EPA was not allowed toproceed with assessments until OMB notified EPA that it had sufficientlyresponded to comments from OMB and other federal agencies. EPA hasalso streamlined its IRIS process, as we recommended in our 2008report, by consolidating some process steps and eliminating others thathad provided opportunities for other federal agencies to suspend IRISassessments to conduct additional research.15

Shortly after it implemented its revised IRIS assessment process in May2009, EPA experienced a surge of productivity in terms of the number of

IRIS assessments it issued. Specifically, from May 2009 throughSeptember 30, 2011, EPA completed 20 IRIS assessmentsmore thandoubling the total productivity it achieved during fiscal years 2007 and2008.16 However, 16 of these were completed in the first year and a half

of implementing the revised process, and productivity fell sharply duringfiscal year 2011, with EPA issuing 4 IRIS assessments (see fig. 2).

15GAO-08-440.

16In fiscal year 2007 EPA issued 2 IRIS assessments. In fiscal year 2008 EPA issued 5assessments4 of which were assessments of related chemicals assessed and peerreviewed together but finalized individually. In addition, in February 20093 monthsbefore the revised IRIS process was announcedEPA issued an assessment fornitrobenzene.

Initial Gains in

Productivity under theRevised Process Have NotBeen Sustained


17/53


Figure 2: Number of Completed IRIS Assessments, Fiscal Years 2002-2011

In completing 4 IRIS assessments in fiscal year 2011, EPA fell significantly

short of its original plan to complete 20 assessmentsa goal that it hadrevised to 9 as of August 2011. In addition, EPA is unlikely to meet its fiscayear 2012 goal of completing 40 assessments.17 As of September 30,

2011, 12 of the 40 assessments that EPA plans to complete in fiscal year2012 are still being drafted (step 1 of the IRIS process). See appendix II forthe status of chemicals in the IRIS assessment process as of September30, 2011. On the basis of the planned time frames EPA established underits revised process, once these 12 IRIS assessments are drafted, EPA willrequire at least 345 days, or 11 months, to complete the remaining IRISprocess steps and issue the assessmentsmaking it unlikely these will becompleted in 2012.

17This goal includes assessments remaining from fiscal year 2011. EPA originally plannedto issue 20 IRIS assessments in fiscal year 2011: 4 were issued, 1 was dropped, 2 weregiven TBDto be determinedcompletion dates, and 13 were added to the fiscal year2012 completion goals.


18/53


The increased productivity occurring after May 2009 does not appear tobe entirely attributable to the revised IRIS assessment process.

According to our analysis of EPA data, the agencys ability to completemore assessments was not due to a fundamental gain in how quicklyassessments are completed, but rather to EPAs ability to clear up thebacklog of assessments that had undergone work under the previousIRIS process and had been delayed for multiple reasons. Most of theassessments completed from May 2009 through September 2011 hadbeen in process 5 years or longer and thus had already passed throughsome key process steps prior to the implementation of the revisedprocess. In addition, most of these completed IRIS assessments were for

standard and moderately complex assessmentsthat is, they were lesschallenging to complete than those for more complex chemicals.Specifically, 17 of 20 assessments issued from May 2009 throughSeptember 30, 2011, were in process for 5 years or longer, and 2 of the20 were for exceptionally complex assessments (see table 2). Forexample, 1 exceptionally complex assessment that EPA did completewas for trichloroethylene (TCE). For information on TCE, as well as onsome other key chemicals for which EPA has not completed IRISassessments, see appendix III.


19/53


Table 2: Completed IRIS Assessments from May 2009 through September 2011

IRIS assessment Level of complexitya

Assessmentcompletion date

Length of time tocomplete assessment

Trichloroacetic acid Moderately complex Sept. 30, 2011 7 years, 9 months

Trichloroethylene (TCE) Exceptionally complex Sept. 28, 2011 13 years, 9 months

Hexachloroethane Standard Sept. 23, 2011 6 years

Urea Standard July 13, 2011 3 years, 6 months

Chloroprene Moderately complex Sept. 30, 2010 11 years, 10 months

Dichloroethylene -1,2-Cis- Standard Sept. 30, 2010 6 years, 8 months

Dichloroethylene -1,2-Trans- Standard Sept. 30, 2010 6 years, 8 monthsPentachlorophenol Moderately complex Sept. 30, 2010 12 years, 9 months

Tetrachloroethane-1,1,2,2 Standard Sept. 30, 2010 5 years

Hydrogen cyanide Standard Sept. 28, 2010 7 years, 6 months

Dioxane-1,4(oral route) Moderately complex Aug. 11, 2010 6 years, 6 months

Carbon tetrachloride Moderately complex Mar. 31, 2010 10 years, 3 months

Ethylene glycol monobutyl ether (EGBE) Moderately complex Mar. 31, 2010 6 years, 2 months

Acrylamide Exceptionally complex Mar. 22, 2010 9 years, 1 month

Bromobenzene Standard Sept. 30, 2009 6 years, 8 months

Thallium Standard Sept, 30, 2009 7 years, 9 months

Trichloropropane-1,2,3 (TCP) Moderately complex Sept, 30, 2009 6 years, 5 months

Cerium Oxide Standard Sept, 29, 2009 4 years, 3 months

Hexanone-2 Standard Sept, 25, 2009 4 years, 8 months

Chlordecone (kepone) Standard Sept, 22, 2009 6 years, 8 months

Average time to complete 7 years, 6 months

Source: GAO analysis of EPA data.

aEPA determines the level of complexity for IRIS assessments.

As of September 30, 2011, EPA had 55 IRIS assessments ongoing and14 on holddown from the 88 assessments that were in various stagesof development when it implemented its revised IRIS assessment process

in May 2009. Since May 2009, EPA has undertaken 6 new assessments,dropped 5 assessments that it determined were no longer required,completed 20 assessments, and continued to have 14 assessments onhold (see table 3). According to EPA officials, assessments that havebeen put on hold will be resumed when the agency has resourcesavailable to staff them.

EPA Has Not SignificantlyReduced Its Ongoing

Assessment Workload,Which Would Enable the

Agency to Start NewAssessments


20/53


Table 3: Ongoing IRIS Assessment Workload Balance

Beginning workload, May 2009 88

New assessments undertakena

6

Dropped from agendab

(5)

Completedc

(20)

On holdd

(14)

Ending workload, September 30, 2011 55


aSince May 2009, EPA has started 3 new IRIS assessments (n-butanol, diethyl phthalate, and

polychlorinated biphenyls [PCBs]noncancer). In addition, EPA decided to split the assessments for 3chemicals (chromium VI, 1,4-dioxane, and methanol), effectively adding 3 assessments to its list ofongoing assessments. It split the latter assessment into assessments of the cancer and noncancereffects of methanol, and the former 2 assessments into assessments of the effects of oral andinhalation exposure.bAs indicated in the Federal Register(75 Fed. Reg. 63827, October 18, 2010), EPA stopped work on(or dropped) the IRIS assessments of perfluorooctane sulfonate-potassium salt (PFOS) andperfluorooctanoic acid-ammonium salt (PFOA) because it is focusing on these chemicals as part of itschemicals management program under the Toxic Substances Control Act (TSCA). In the sameFederal Registernotice, EPA announced that it had stopped work on the IRIS assessment ofweathered toxaphene because of lack of data to support an IRIS assessment, and that it had revisedits approach to the IRIS assessment of asbestos, deciding to focus exclusively on a certain type ofasbestosLibby amphibole asbestosin order to respond to the needs of the agency and the miningcommunity of Libby, Montana. Under the revised approach, EPA dropped its plans to assessasbestos other than Libby amphibole asbestos. In August 2011, EPA decided to drop a fifthassessment, the draft assessment of mirex, because of its relatively low priority EPA-wide.c

For a list of the 20 completed IRIS assessments from May 2009 through September 2011, see table 2.dThe following 14 assessments remain on hold as of September 30, 2011: manganese, ethylene

dichloride, tungsten, tertiary-amyl-methyl ether (TAME), ethylbenzene, alkylates, antimony, carbonylsulfide, diisopropyl ether (DIPE), tertiary amyl ethyl ether (TAEE), bisphenol A, refractory ceramicfibers, isopropanol, and ethanol.

However, this tally of IRIS assessments does not reflect the true extent ofEPAs workload or the backlog of demand for IRIS assessments. Beyondthe 55 ongoing IRIS assessments and 14 on hold, the demand foradditional IRIS assessments is unclear. With existing resources devotedto addressing its current workload of ongoing assessments, EPA has notbeen in a position to routinely start new assessments. In late 2010, for thefirst time since 2007, EPA solicited nominations for new IRIS

assessments from EPA program and regional offices, as well as from thepublic and federal agencies that participate in IRIS interagency reviews.However, as of September 30, 2011, EPA officials had not decided whichchemicals to include on the IRIS agendaand thus include in their


21/53


workload.18 Moreover, instead of nominating new chemicals for

assessment in 2010, one regional office requested that the IRIS Programfocus its efforts on completing assessments currently under way. Inaddition, in 2007, the Office of Air and Radiationwhich developsnational programs, policies, and regulations for controlling air pollutionand radiation exposurerequested that ongoing assessments beexpedited for 28 chemicals that it identified as high-priority and requiredto fulfill its regulatory mandates. As of September 30, 2011, 17 of the 28assessments the office identified are ongoing, and 3 are on hold. Seeappendix IV for EPAs expected completion dates for IRIS assessmentscurrently in the assessment process.

In addition, other assessments in the IRIS database may need to beupdated. As we reported in March 2008, EPA data from 2001 through2003 indicated that 287 of the assessments in the IRIS database at thattime may need to be updated. In October 2009, EPA announced in theFederal Registerthe establishment of the IRIS Update Project. The statedpurpose of the project was to update IRIS toxicity values, such as oralreference doses or inhalation reference concentrations, that are morethan 10 years old. However, according to EPA officials, since the projectwas announced, little progress has been made toward updating theseassessments. We note that even if EPA were to overcome the significant

productivity difficulties it has experienced in recent years and meet itsgoal of completing 40 assessments in fiscal year 2012, it is not clear thatthis level of productivity would meet the needs of EPA program officesand other users.

IRIS assessments have taken longer than the time frames establishedunder the revised IRIS process. Since implementing the revised process,most IRIS assessments have exceeded the established time frames foreach step of the process. EPA officials, however, told us that the timeframes established for the steps in the revised IRIS assessment processapply only to standard assessmentsand not to moderately or

exceptionally complex assessments. While EPA officials have said thatthey are trying to hold moderately complex assessments to the establishedtime frames, EPA does not have a written policy that describes theapplicability of these time frames or written criteria for designating IRIS

18The IRIS agenda lists chemicals that are to be assessed during a given fiscal year.

EPA Has Not MetEstablished Time Framesfor IRIS AssessmentProcess Steps


22/53


assessments as standard, moderately complex, or exceptionally complex.Consequently, it is unclear how IRIS users will know which assessmentsare standard, moderately complex, or exceptionally complex and what timeframes will be required to complete them.

According to EPA officials, NCEA management, including IRIS Programmanagement, is tracking the time it takes for each IRIS assessment tocomplete the various steps in the IRIS process. However, EPA has notyet analyzed these data to determine whether the time framesestablished for each step or the overall 23-month process are realistic.

According to EPA officials, they do not yet have the data needed to draw

conclusions regarding completion time frames.

On the basis of our analysis of EPA data, however, we determined howlong each IRIS process step was taking on average compared with thetime frames established for each step under the May 2009 revisedprocess. We performed this analysis for the 55 assessments that wereongoing, as of September 30, 2011, and the 20 assessments that werecompleted after May 2009. Because none of the 20 IRIS assessmentscompleted from May 2009 through September 2011 were initiated afterthe revised process was implemented, it was not possible to fully evaluatethe extent to which EPA is adhering to the new 23-month time frame.Further, we combined our analysis of steps 4 and 5 because EPA data donot indicate when step 4 ends and when step 5 begins, and we combinedsteps 6 and 7 for the same reason. According to our analysis, onaverage, assessments of all types have taken longer than the establishedtime frames for every step in the IRIS process (see table 4).


23/53


Table 4: May 2009 IRIS Process Step Time Frames and Actual Completion Times for Each Step

Average completion timea

(Number of assessments that completed the step)

IRIS process steps and establishedEPA time frames

bStandard assessments

Moderately complexassessments

Exceptionally complexassessments

Step 1IRIS draft assessmentcompleted

c

(345 days)

675 days

(1 assessment)

352 days

(1 assessment)

506 days

(1 assessment)

Step 2Internal agency review

(60 days)

91 days

(4 assessments)

141 days

(2 assessments)

109 days

(4 assessments)

Step 3EPA-led interagency scienceconsultation

(45 days)

73 days(8 assessments)



Step 4External peer review, andStep 5Draft assessment revised

(165 days combinedd)

294 days

(6 assessments)

442 days

(4 assessments)

523 days

(2 assessments)

Step 6aInternal agency review,

Step 6bEPA-led interagency scienceconsultation and discussion, and

Step 7Final IRIS assessment posted

(75 days combinede)

80 days

(11 assessments)

92 days

(7 assessments)

155 days

(2 assessments)


a

We calculated average completion times, rounded to the nearest day, using EPA-provided data. Inour calculations, we considered only assessments that began and completed a step under the May2009 process. This included the 55 ongoing assessments and the 20 completed since May 2009. Forexample, some assessments completed multiple steps since May 2009, while others completed onlyone or two.bAccording to EPA officials, the time frames established for the steps in the IRIS assessment processapply to standard assessments. EPA officials told us they are trying to hold moderately complexassessments to the 23-month time frame, and the process time does not apply to exceptionallycomplex assessments.cAs of September 30, 2011, 23 other assessments were still in step 1, and 21 of these had alreadyexceeded the 345-day time frame for the step. These 21 assessments had been started prior toEPAs implementation of the May 2009 IRIS process.dWe combined steps 4 and 5 because the EPA data do not indicate when step 4 ends and when step

5 begins.eWe combined steps 6 and 7 because the EPA data do not indicate when step 6 ends and when step

7 begins.

Some other federal agencies that participate in interagency reviewsexpressed concern that in some cases time and resource constraintspresent challenges as they try to meet EPAs time frames for the twointeragency review steps. In addition to the time limits established under therevised process, in an effort to increase productivity and complete more IRISassessments, EPA officials said that, beginning in April 2011, the agency


24/53


began to accelerate the number of draft assessments sent through theinteragency review steps. However, officials from other federal agenciesincluding HHS and DODtold us that they have advised EPA that theaccelerated pace of interagency reviews in the second half of fiscal year2011 strained their resources. In addition, the official from NASA told us thatnot only are the increased pace of reviews straining the agencys resources,but that it has also affected the ability to provide in-depth independenttechnical reviews and interagency comments. EPA officials also told us thatthe interagency reviewer at NASA is so concerned with the pace of theinteragency reviews under the revised process that NASA officials haveasked OMB to form an interagency work group to discuss the reviews.

EPA faces both long-standing and new challenges in implementing theIRIS Program. First, the National Academies has identified recurringissues with how the IRIS Program develops and presents itsassessments and has suggested improvements. Second, EPA has notconsistently provided reliable information on ongoing and planned IRISassessments to IRIS users. Third, unresolved discussions with OMBregarding EPAs responses to Data Quality Act challenges may impedeEPAs ability to issue completed IRIS assessments.

The National Academies and EPAs Science Advisory Board haveidentified several recurring issues with how EPA develops and presentsIRIS assessments. For example, in April 2011, the National Academies inits independent scientific review of EPAs draft IRIS assessment offormaldehyde provided a critique of EPAs development and presentationof draft IRIS assessments. Overall, the National Academies noted somerecurring methodological problems in the draft IRIS assessment offormaldehyde.19 In addition, in the report the National Academies also

identified recurring issues concerning clarity and transparency with EPAsdevelopment and presentation of its draft IRIS assessments.

The National Academies and Science Advisory Board have identifiedsimilar clarity and transparency issues in peer review reports over the

19We did not evaluate the scientific content or quality of IRIS assessments.

EPA Faces Long-standing and NewChallenges inImplementing theIRIS Program

The National AcademiesHas Identified RecurringIssues with How the IRISProgram Develops andPresents Its Assessments


25/53


past 5 years.20 Some of these reports stated that EPA should more clearly

explain its reasons for including or excluding the scientific studiessupporting draft IRIS assessments. In addition, some reports stated thatEPA should more transparently present its justifications for itsmethodological approaches. Independent of its review of theformaldehyde assessment, the National Academies also provided aroadmap for revision that made suggestions for improvements to theIRIS draft development process, during which EPA selects and evaluatesevidence (the literature search) and drafts an assessment (step 1). TheNational Academies roadmap for revision suggested that EPA take thefollowing steps, among others:

use clear, standardized methods to identify and select study evidence;

use a standardized approach to evaluate and describe studystrengths and weaknesses and the weight of evidence, describe and

justify the assumptions and models used, and adopt a standardizedapproach to characterizing uncertainty factors;21 and

present methodology and findings more clearly and more conciselythrough better use of graphics and tables and use a template tofacilitate a consistent description of the approach to study selection.

The National Academies report on the draft IRIS assessment offormaldehyde specifically noted that EPA should not delay the finalizationof the assessment in order to implement any of the suggestions it maderegarding the overall IRIS process. As of September 30, 2011, accordingto EPA officials, the agency is revising the assessment in response to theNational Academies suggestions, but the status page on EPAs websitefor formaldehyde lists TBDto be determinedas the posting date forthe final assessment.

20In addition to reviewing the National Academies peer review report on the draft IRIS

assessment of formaldehyde, we reviewed National Academies peer review reports on thedraft assessments of tetrachloroethylene (2010) and dioxin (2006) and the Science

Advisory Board peer review reports on the draft assessments of trichloroethylene (TCE)(2011), acrylamide (2008), ethylene oxide (2007), dioxin (2011), polycyclic aromatichydrocarbon (PAH) mixtures (2011), and inorganic arsenic (2011).

21EPA uses uncertainty factors in the derivation of IRIS values to account for uncertaintydue to, among other things, variability in susceptibility to a chemical among humans, theextrapolation of animal study data to humans, and the extrapolation of a chemicals effectsover a lifetime.


26/53


In July 2011, EPA announced that it planned to respond to the NationalAcademies suggestions by implementing changes to the way it developsdraft IRIS assessments. In announcing the planned changes, EPA statedthat it would take the following actions:

enhance its approach to identifying and selecting scientific studyevidence;

provide more complete documentation of its approach to evaluatingscientific study evidence and indicate which criteria were mostinfluential in its evaluation of the weight of evidence; and

concisely state the criteria used to include or exclude studies,continue to use existing IRIS guidelines to enhance the clarity andtransparency of its data evaluation and presentation of findings andconclusions, eliminate the need for some report text usingstandardized tables, and portray toxicity values graphically.

According to EPA officials, in implementing these changes, EPA willsubject those assessments that are in earlier stages of development tomore extensive changes than those in later stages of development. It willchange the latter as feasible without repeating steps in the overall IRISprocess. However, EPA has not provided a more detailed description of

how the National Academies suggestions will apply to each of theassessments in its current inventory of IRIS assessments. Without a moreprecise description of which drafts would be considered in the earlierstages of development or what more extensive changes would entail, itis too soon to provide a comprehensive assessment of EPAs approach.In addition, it is not transparent to stakeholders and other interestedparties which assessments will be subject to these changes and whichwill not. EPA established the Board of Scientific Counselors (BOSC), anadvisory committee composed of non-EPA technical experts fromacademia, industry, and environmental communities, to provideindependent advice, information, and suggestions to the Office ofResearch and Development (ORD) research programwhich houses theIRIS Program.22 Part of BOSCs mission is to evaluate and provide advice

concerning the utilization of peer review within ORD to sustain and

22Members of the BOSC Executive Committee and subcommittees constitute adistinguished body of scientists and engineers who are recognized experts in theirrespective fields.


27/53


enhance the quality of science in EPA. It is unclear if BOSC will have arole in reviewing EPAs response to the National Academies suggestions

We reviewed two IRIS assessmentsone completed and one still in draftformthat reflect changes EPA has made in response to the National

Academies suggestions.23 First, for its assessment of urea, finalized in

July 2011, EPA streamlined the report by moving sections of text from thebody to an appendix, which shortened the body of the assessment from89 to 57 pages, making it more concise. In addition, we reviewed the draftIRIS assessment of diisobutyl phthalate (DIBP), which EPA provided tous, that was undergoing agency review (step 2) and reflects some of the

National Academies suggestions regarding presentation. For example, itincludes (1) descriptive and pictorial explanations of the study selectionmethods used; (2) tables that, among other things, give side-by-sidecomparisons of studies considered in determining the oral reference dosefor the chemical; and (3) brief descriptions of the strengths andweaknesses of various studies considered. For these two assessments, itappears that EPA has begun to enhance the readability of itsassessments by making changes that appear to be in line with thesuggestions made by the National Academies.

EPA uses two primary mechanismsthe IRIS agenda and a websitefeature known as IRISTrackto make information on the status of IRISassessments available to EPA program and regional offices, other federalagencies, and the public.24 EPA has not effectively used these two

mechanisms, or a third that we recommended in March 2008that theagency provide a 2-year notice of its intent to assess specific chemicalsto consistently provide reliable information on IRIS assessments tostakeholders and other interested parties.25

23According to EPA, these two assessments were changed based on the National

Academies suggestions for improvement. We chose one assessment because it wascompleted during our review and EPA provided us the other assessment, which it saidreflected changes made based on the National Academies suggestions. These twoassessments may not be representative of all assessments, but are examples ofassessments that EPA has changed based on the National Academies suggestions.

24EPAs IRIS assessment tracking system is formally called the IRIS SubstanceAssessment Tracking System.

25GAO-08-440.

EPA Has Not ConsistentlyProvided ReliableInformation on Plannedand Ongoing IRIS

Assessments to IRIS Users


28/53


First, EPA has not published an IRIS agenda in the Federal Registeridentifying the chemicals that EPA plans to assess (both new andongoing assessments)since it announced its 2008 IRIS agenda inDecember of 2007. EPA started developing an annual IRIS agenda andproviding it to the public in a notice in the Federal Registerin 1997.26 In

late 2010, EPA began to solicit nominations for its fiscal year 2011 IRISagenda from its program and regional offices, as well as from the publicand federal agencies that participate in IRIS interagency reviews.However, as of September 30, 2011, EPA had not published its fiscalyear 2011 agenda. In addition, some of the information provided in theFederal Registernotices about the IRIS agenda has been incomplete. For

example, an October 2010 Federal Registernotice contained a list ofchemicals currently on the IRIS agenda but did not distinguish betweenchemicals the agency was actively assessing and those it had designatedfor future assessment.27 We reported on similar issues in March 2008

noting that EPA had identified some assessments that had beensuspended as ongoing.28

Second, EPA has not kept information on the status of the individualongoing assessments up to date in IRISTrackan issue we also reportedon in 2008.29 EPAs IRISTrack, a feature of its website, is intended to

provide stakeholders and other interested parties with information on draft

IRIS assessmentsspecifically, estimated start and end dates for stepsin the IRIS process.30 For example, officials from the Office of Water

indicated that that their office relies heavily on IRISTrack for informationabout the status of IRIS assessments. In addition to not updatingIRISTrack, EPA recently removed some key information presented inIRISTrack. Now, in some cases, the IRISTrack date for the beginning ofdraft development (step 1) understates the actual duration of anassessmentsometimes by many years. For example, IRISTrackindicates that draft development for the dioxin assessment began in the

26EPA also uses the Federal Registerto announce other IRIS-related developments, suchas public meetings of peer review panels and public listening sessions, at which interestedparties are invited to give comments on draft IRIS assessments.

27Federal Register(75 Fed. Reg. 63827, October 18, 2010).

28GAO-08-440.

29GAO-08-440.

30IRISTrack provides estimated start and end dates by fiscal year quarter.


29/53


first quarter of fiscal year 2009; in fact, as we have reported, EPA hasbeen assessing dioxin since 1991.31 IRISTrack also understates the

duration of assessments of other chemicals of key concernforformaldehyde, naphthalene, and TCE. Therefore, current and accurateinformation regarding when an assessment will be started, whichassessments are currently ongoing, and when an assessment isprojected to be completed is presently not publicly available.

Third, EPA does not provide at least 2 years notice of its intent to assessspecific chemicals, as we recommended the agency should do in ourMarch 2008 report to give agencies and other interested parties the

opportunity to conduct research needed to fill any data gaps.32 In

commenting on our report, EPA agreed to consider our recommendation,and EPA officials recently stated that they continue to agree with it, but asof September 30, 2011, the agency still had not taken steps to implementour recommendation.

Discussions between EPA and OMB officials regarding Data Quality Actchallenges related to specific draft IRIS assessments have been ongoingfor over a year without resolution. If these unresolved discussionscontinue, they could contribute to delays of IRIS assessments. According

to EPA officials, OMB would like to return to its role in the priorassessment process, in which it managed interagency reviews and madethe final determination as to whether EPA has satisfactorily responded tocomments from OMB and officials in other federal agencies.

The Information Quality Act, commonly called the Data Quality Act,requires OMB to issue governmentwide guidelines to ensure andmaximize the quality, objectivity, utility, and integrity of information,including statistical information, disseminated to the public.33 In addition, it

required agencies to issue their own guidelines, set up administrativemechanisms to allow affected parties to seek the correction of informationthey considered erroneous, and report periodically to OMB information

about Data Quality Act challenges (requests for correction of agency

31GAO-08-440.

32GAO-08-440.

33Consolidated AppropriationsFiscal Year 2001, Pub. L. No. 106-554, 515, 114 Stat.2763A-153 to 2763A-154 (2000) (44 U.S.C. 3516 note).

Unresolved Discussionsbetween EPA and OMBCould Contribute to Delays


30/53


information) and how the agencies addressed them.34 Under its data

quality guidelines, when EPA provides opportunities for public participationby seeking comments on information, such as during a rulemaking, theagency uses the public comment process rather than EPA guidelines toaddress concerns about EPAs information. This is consistent with OMBsdata quality guidelines, which encourage agencies to incorporate dataquality procedures into their existing administrative practices rather thancreate new and potentially duplicative or contradictory processes.

According to EPAs data quality guidelines, the public comment periodserves the purposes of the guidelines, provides an opportunity forcorrection of information, and does not duplicate or interfere with the

orderly progression of draft documents through an established processinthis case, the IRIS assessment process. That is, the external peer reviewand associated public comment period provide the public with theopportunity to raise questions regarding the quality of the information beingused to support an IRIS assessment. According to EPA officials, federalagency responses to data quality challenges must be cleared by OMBbefore EPA sends responses to the parties filing challengesalthough nolaw or guidance specifically provides for such reviews.

In June and July 2010, EPA received Data Quality Act challengesregarding two draft IRIS assessments. According to EPA officials, in its

draft responses to these data quality challenges, EPA declined to reviewthe challenged data because, according to agency policy, draft IRISdocuments are not subject to data quality challenges. EPA used the sameapproach in 2006 when responding to and declining a similar challengeregarding a draft IRIS assessment; at that time, OMB approved the EPAresponse. EPA sent its draft responses for the two more recentchallenges to OMB for approval in September 2010 and January 2011.EPAs data quality guidelines set a goal of responding to Data Quality Actchallenges within 90 days, but EPA officials said that they still await adecision by OMB. According to EPA officials, OMB is delaying a decisionbecause OMB would like to return to its role in the prior assessmentprocess, in which it managed interagency reviews and made the final

34OMBs data quality guidelines recognize that information quality comes at a cost, andthat agencies should weigh the costs and benefits of increasing information quality. 67Fed. Reg. 8453 (February 22, 2002). OMB has stated that its involvement in the IRISprocess increases the quality of the assessments, but it has produced no cost-benefit orother analysis supporting that statement, nor has it disclosed the performance measures ituses to evaluate assessment quality.


31/53


determination as to whether EPA has satisfactorily responded tocomments from OMB and officials in other federal agencies.

EPA officials told us that as of September 30, 2011, the issues regardingdata quality challenges had not delayed the progress of draft IRISassessments.35 Meanwhile, OMB staff told us that they had sent

comments to EPA on the draft responses and await EPAs reply to theircomments. It appears to GAO that the discussions of these issuesbetween EPA and OMB officials, which have been ongoing for over ayear without resolution, have highlighted the agencies differencesregarding the revised IRIS process. If these differences persist, they

could contribute to the compounding effects of delays in the IRIS process,discussed here and in our earlier work. For example, in August 2011,EPA received a third data quality challenge on an assessment that EPAhad expected to be finalized at the end of fiscal year 2011.36 For reasons

that remain unclear, EPA now projects that this assessment will not befinalized until fiscal year 2012. We note that the assessment had enteredthe interagency science discussion (step 6b) in July 2011. EPA askedinteragency reviewers to submit written comments by August 26, 2011,but as of September 2011, OMB reviewers have not yet submittedcomments.

35One of the challenged assessments, the draft inorganic arsenic cancer assessment, iscurrently undergoing post-peer review EPA revisions (step 5). The other challengedassessment, the draft cancer assessment of methanol, is currently on hold while EPAreviews some of the data underlying the findings of the draft assessment.

36On August 1, 2011, the International Platinum Group Metals Association filed a requestfor correction on the IRIS assessment for Halogenated Platinum Salts and PlatinumCompounds. In addition, The National Academies has observed that reaching consensuson all aspects of the scope and conduct of a risk assessment among decision-makers and

stakeholders representing diverse interests will not always be feasible. In addition, it is notnecessarily in the public interest to delay the risk assessment where consensus is difficultto achieve. Science and Decisions: Advancing Risk Assessment(Washington, D.C.: TheNational Academies Press, 2009). Moreover, EPA has noted in its data quality guidelinesthat most environmental statutes obligate EPA to act to prevent adverse environmentaland human health impacts. For many of the risks that we must address, data are sparseand consensus about assumptions is rare. In the context of data quality, we seek to strikea balance among fairness, accuracy, and efficient implementation. Refusing to act untildata quality improves can result in substantial harm to human health, safety, and theenvironment.


32/53


The IRIS process reforms EPA began implementing in May 2009 haverestored EPAs control of the process and increased its transparency.Notably, EPA has addressed concerns we raised in our March 2008report regarding the transparency of comments from both the interagencyscience consultation and discussion steps in the IRIS process. Makingthese comments publicly available is especially important becauseagencies providing input may have a vested interest in the outcome of theassessment should it lead to regulatory or other actions. As a result,stakeholders, including EPA regional and program offices, the public, andindustry, can now see which other federal agencies comment and thenature of their comments, making IRIS assessments more transparent. In

addition, EPA now manages the interagency science consultation anddiscussion steps and has streamlined the IRIS process.

Progress in other areas, however, has been more limited. For example,even for its less challenging assessments, EPA took longer than itsestablished time frames for accomplishing steps in the revised processcalling into question the feasibility and appropriateness of the establishedtime frames in the IRIS assessment process for standard assessments.Thus, the established time frames may not be feasible. It is also unclearwhether the established time frames apply to moderately complexassessments because EPA does not have a written policy that describesthe applicability of the time frames, although EPA officials said they aretrying to hold moderately complex assessments to the 23-month timeframe. Similarly, EPA does not have written criteria for designating IRISassessments as standard, moderately complex, or exceptionally complex.We note that EPA has not analyzed the time frames to determine whetherthe actual time taken for each step of the overall 23-month process isrealistic. Such an analysis would provide more accurate information forEPA to use in establishing time frames for these assessments. Nothaving established time frames for these assessments also createsuncertainty for many stakeholders with significant interest in IRISassessments.

EPA also faces both long-standing and new challenges in implementingthe IRIS Program. Notably, the National Academies and Science AdvisoryBoard have identified recurring issues of clarity and transparency of draftIRIS assessments. Consequently, as part of its independent scientificreview of EPAs draft IRIS assessment of formaldehyde, the National

Academies also provided suggestions in a roadmap for revision thatincluded suggestions for improving EPAs development and presentationof draft IRIS assessments in general. The report identified recurringmethodological issues with how the IRIS Program develops and presents

Conclusions


33/53


its assessments and suggested improvements. EPA announced that itintends to address the issues raised in the National Academies report buthas not publicly indicated how these proposed changes would be appliedto its current inventory of IRIS assessments. Many of the issues raised inthe National Academies report have been brought to the agencysattention previously. It is unclear whether any independent entity withscientific and technical credibility, such as EPAs Board of ScientificCounselors, will have a role in reviewing EPAs planned response to theNational Academies suggestions to ensure that EPA addresses theselong-standing issues.

In addition, EPA has not addressed other long-standing issues regardingthe accuracy and availability of information on the status of IRISassessments to IRIS usersincluding stakeholders such as EPA programand regional offices, other federal agencies, and the public. For example,since 2007, EPA has not published in the Federal Registeran IRIS agendathat includes information on chemicals the agency is actively assessing orwhen it plans to start assessments of other listed chemicals. The agencyalso has not updated IRISTrack to display all current information on thestatus of assessments on the IRIS agenda, including estimated start datesand end dates of steps in the IRIS process. In addition, EPA has recentlyremoved some key information presented in IRISTrack that showed theduration of IRIS assessments. Now, in some cases, the IRISTrack date forthe beginning of draft development underestimates the actual duration ofan assessmentsometimes by many years. Therefore, current andaccurate information regarding when an assessment will be started, whichassessments are currently ongoing, and when an assessment is projectedto be completed is presently not publicly available. Finally, as werecommended the agency should do in our March 2008 report, EPA doesnot provide at least 2 years notice of its intent to assess specific chemicalswhich would give agencies and other interested parties the opportunity toconduct research needed to fill any data gaps.

To improve EPAs IRIS assessment process, we are making the followingsix recommendations:

To better ensure the credibility of IRIS assessments by enhancing theirtimeliness and certainty, we recommend that the EPA Administratorrequire the Office of Research and Development to

assess the feasibility and appropriateness of the established timeframes for each step in the IRIS assessment process and determine

Recommendations forExecutive Action


34/53


whether different time frames should be established, based oncomplexity or other criteria, for different types of IRIS assessments, and

should different time frames be necessary, establish a written policythat clearly describes the applicability of the time frames for each typeof IRIS assessment and ensures that the time frames are realistic andprovide greater predictability to stakeholders.

To better ensure the credibility of IRIS assessments by enhancing theirclarity and transparency, we recommend that the EPA Administratorrequire the Office of Research and Development to submit for

independent review to an independent entity with scientific and technicalcredibility, such as EPAs Board of Scientific Counselors, a plan for howEPA will implement the National Academies suggestions for improvingIRIS assessments in the roadmap for revision presented in the National

Academies peer review report on the draft formaldehyde assessment.

To ensure that current and accurate information on chemicals that EPAplans to assess through IRIS is available to IRIS usersincludingstakeholders such as EPA program and regional offices, other federalagencies, and the publicwe recommend that the EPA Administratordirect the Office of Research and Development to

annually publish the IRIS agenda in the Federal Registereach fiscalyear;

indicate in published IRIS agendas which chemicals EPA is activelyassessing and when EPA plans to start assessments of the otherlisted chemicals; and

update IRISTrack to display all current information on the status ofassessments of chemicals on the IRIS agenda, including projectedand actual start dates, and projected and actual dates for completionof steps in the IRIS process, and keep this information current.

We provided a draft of this report to the Administrator of EPA for reviewand comment. In written comments, which are included in appendix V, EPAagreed with the reports recommendations. EPA also provided technicalcomments, which we incorporated into the report as appropriate.Specifically, EPA agreed that it should (1) assess the feasibility andappropriateness of the established time frames for each step in the IRISassessment process by using available program performance measures

Agency Commentsand Our Evaluation


35/53


collected since the current IRIS process was established to evaluatedetermine whether different time frames should be established, based oncomplexity or other criteria, for different types of IRIS assessments, (2)determine if different time frames are necessary, establish a written policythat clearly describes the applicability of the time frames for each type ofIRIS assessment and ensures that the time frames are realistic and providegreater predictability to stakeholders, (3) continue to implement the 2011suggestions for improving IRIS assessments in the roadmap for revisionpresented in the National Academies peer review report on the draftformaldehyde assessment and seek independent review through theScience Advisory Board to ensure that the agency is addressing the

recommendations, (4) annually publish the IRIS agenda in the FederalRegistereach fiscal year, (5) indicate in published IRIS agendas whichchemicals EPA is actively assessing and when EPA plans to startassessments of the other listed chemicals, and (6) update IRISTrack todisplay all current information on the status of assessments of chemicalson the IRIS agenda, including projected and actual start dates, andprojected and actual dates for completion of steps in the IRIS process, andkeep this information current.

As agreed with your office, unless you publicly announce the contents ofthis report earlier, we plan no further distribution until 30 days from thereport date. At that time, we will send copies to the Administrator of EPA,the appropriate congressional committees, and other interested parties. Inaddition, the report will be available at no charge on the GAO website athttp://www.gao.gov.

If you or your staff members have any questions about this report, pleasecontact me at (202) 512-3841 [email protected]. Contact points forour Offices of Congressional Relations and Public Affairs may be foundon the last page of this report. GAO staff who made major contributions tothis report are listed in appendix VI.

Sincerely yours,

David C. TrimbleDirectorNatural Resources and Environment
http://www.gao.gov/http://www.gao.gov/mailto:[email protected]:[email protected]:[email protected]:[email protected]://www.gao.gov/


36/53

Appendix I: Scope and Methodology


This appendix details the methods we used to assess the EnvironmentalProtection Agencys (EPA) management of its Integrated Risk InformationSystem (IRIS). For this review, our objectives were to evaluate (1) EPAsprogress in completing IRIS assessments under the May 2009 process and(2) the challenges, if any, that EPA faces in implementing the IRIS Program.

To address these objectives, we reviewed relevant EPA documents,including documents outlining the April 2008 and the May 2009 versions ofthe IRIS assessment process; documents related to IRIS performancemetrics; chemical nomination forms submitted by EPA regional andprogram offices, federal agencies, and others; and documents and other

information on the public EPA website, including the IRIS database andIRISTrack, the assessment tracking system available at the IRIS website.In addition, we reviewed other relevant documents, including FederalRegisternotices announcing, among other things, IRIS agendas, as well asdocuments related to EPAs meetings with other federal agencies involvedin interagency reviews of draft IRIS assessments. We did not evaluate thescientific content or quality of IRIS assessments; however, we did reviewthe National Academies peer review report on the draft IRIS assessmentof formaldehyde to evaluate their suggestions for overall improvements tothe development of IRIS assessments and other peer review reports by theNational Academies and EPAs Science Advisory Board to evaluate theirsuggestions for improvements to draft IRIS assessments. In addition, weinterviewed officials from EPAs National Center for Environmental

Assessment (NCEA) who manage the IRIS Program, including the ActingCenter Director, the Associate Director for Health, and the IRIS Program

Acting Director, to obtain their perspectives on, among other things, theMay 2009 IRIS process and the effects of changes from the April 2008 IRISprocess, the extent to which EPA has made progress in completing timely,credible chemical assessments, challenges EPA faces in completingassessments, and EPAs process for responding to Data Quality Actchallenges. We interviewed officials from EPAs Office of EnvironmentalInformation to obtain their perspectives on EPAs process for responding todata quality challenges. We also attended two Board of Scientific

Counselors (BOSC) meetings to understand the boards role in providingadvice, information, and recommendations about the Office of Researchand Development (ORD) research programs, including IRIS.

For the first objective, we obtained and analyzed data from fiscal year1999 through September 30, 2011, including data, spreadsheets, projectplans, and other documents used in IRIS assessment planning,development, and completion. From the data we gathered, we analyzedinformation on IRIS productivity, including information on the number of



37/53



IRIS assessments completed and initiated, the status of IRISassessments that are currently in progress or on the IRIS agenda, andthe completion dates and durations of IRIS assessment process stepscompleted or currently in progress for given chemical assessments. Inaddition, we assessed the reliability of the data we received from EPA forour first objective. Our assessment consisted of interviews and e-mailexchanges with EPA officials about the data system, the method of datainput, and internal data controls and documentation, among other areas.We also corroborated the data with other sources, where possible. Forexample, we verified the information provided in tables of IRISassessment start dates and completion dates of IRIS assessment

process steps through interviews and e-mail exchanges with the NCEAofficials responsible for maintaining these data. Through our assessment,we determined that the data were sufficiently reliable for our purposes.

For the second objective, we interviewed the chair of the NationalAcademies Committee to Review EPAs Draft IRIS Assessment ofFormaldehyde to obtain his perspective on the National Academiessuggestions for improvements to the IRIS assessment process. Weinterviewed officials from the Office of Management and Budgets (OMB)Office of Information and Regulatory Affairs (OIRA) to obtain theirperspectives on interagency review of draft IRIS assessments, OMBsprocess for responding to EPA with regard to Data Quality Act challenges,and OMBs process for reviewing and approving EPA guidance documents.In addition, we interviewed officials from the Department of Defense (DOD),the National Aeronautics and Space Administration (NASA) and theDepartment of Health and Human Services (HHS)includingrepresentatives from the Centers for Disease Control and PreventionsNational Center for Environmental Health (NCEH)/ Agency for ToxicSubstances and Disease Registry (ATSDR), National Institute forOccupational Safety and Health (NIOSH). We also interviewed HHS officialsfrom the Food and Drug Administration (FDA); the National Institute ofEnvironmental Health Sciences/National Toxicology Program and the Officeof the Secretary. We also interviewed representatives from a chemical

industry group and a nonprofit research and educational organization.

We conducted this performance audit from July 2010 to December 2011in accordance with generally accepted government auditing standards.Those standards require that we plan and perform the audit to obtainsufficient, appropriate evidence to provide a reasonable basis for ourfindings and conclusions based on our audit objectives. We believe thatthe evidence obtained provides a reasonable basis for our findings andconclusions based on our audit objectives.


38/53

Appendix II: Status of Chemicals in the IRISAssessment Development Process, as ofSeptember 30, 2011


Step 1Draftdevelopment(24 assessmentsin step)

Step 2EPAinternal review(8 assessmentsin step)

Step 3Interagencyscienceconsultation(2 assessmentsin step)

Step 4Externalpeer review andpublic comment(9 assessments instep)

Step 5EPAdraft revision(7 assessmentsin step)

Step 6a andbFinal EPAreview /Interagencysciencediscussion(1 assessmentin step)

Step 7Completion andposting(4 assessmentsin step)

Acetaldehyde

Arsenic, inorganic(noncancer)

Beryllium (cancer)

Butanol, t-Cadmium

Chloroethane

Chromium VI(inhalation)

Cobalt

Copper

Di(2-ethylhexyl)adipate (DEHA)

Ethyl tertiary butylether (ETBE)

Hexabromo-cyclododecane

Hexachloro-butadiene

Hexahydro-1,3,5-trinitro-1,3,5-triazine (RDX)

Methyl tert-butylether (MTBE)

Naphthalene

Nickel (soluble salts)

Phthalates(cumulative)

Phthalates:

-Dibutyl (DBP)

-Di(2-ethylhexyl)(DEHP)

-Diethyl (DEP)

Styrene

Uranium

Vinyl acetate

Ammonia

Chloroform

Phthalates:

-Butyl benzyl (BBP)

-Diisobutyl (DIBP)-Diisononyl (DINP)

-Dipentyl (DPP)

Trimethylbenzene,1,2,4-

gao - 2012 epa challenges remain with integrated risk information system program

Documents